Differential pixel circuit and method of computer vision applications

ABSTRACT

A pixel cell for differential light sensing includes a plurality of photodiodes and a corresponding plurality of storage diodes. Each storage diode is disposed between a first adjacent photodiode and a second adjacent photodiode, and each storage diode is configured to receive photo charges from either or both of the first adjacent photodiode and the second adjacent photodiode. Each photodiode is disposed between a first adjacent storage diode and a second adjacent storage diode, and each photodiode is configured to transfer photo charges to either or both of the first adjacent storage diode and the second adjacent storage diode.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/598,390, filed on Dec. 13, 2017, the contents ofwhich are hereby incorporated by reference in their entirety.

The following regular U.S. patent applications (including this one) arebeing filed concurrently, and the entire disclosure of the otherapplications are incorporated by reference into this application for allpurposes:

-   -   U.S. patent application Ser. No. 16/219,829, filed on Dec. 13,        2018, entitled “GLOBAL SHUTTER PIXEL CIRCUIT AND METHOD FOR        COMPUTER VISION APPLICATIONS;” AND    -   U.S. patent application Ser. No. 16/219,847, filed on Dec. 13,        2018, entitled “DIFFERENTIAL PIXEL CIRCUIT AND METHOD OF        COMPUTER VISION APPLICATIONS.”

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to image sensorpixel circuits and methods for time-of-flight (ToF) depth measurementsystems.

BACKGROUND OF THE INVENTION

Image sensors are used in a wide range of applications. Examples includedigital cameras, mobile phones, home appliances, endoscopes, andsatellite telescopes. Some image sensors are implemented usingcomplementary metal-oxide semiconductor (CMOS) technology. In thesesensors, the number of MOS transistors equal the number of pixels. Thetransistors are used to convert optical images to electrical signals.

A number of methods may be used to drive the pixel circuits of a CMOSimage sensor. Examples include a rolling shutter (RS) method and aglobal shutter (GS) method. In the rolling shutter method, signals arephoto-electrically converted by photo elements in each row in one frame.The signals are transferred to one or more floating diffusion nodes ineach row that is sequentially selected, and an image signal of acorresponding pixel is output. In the global shutter method, all signalsare photo-electrically converted by all photo elements in one frame. Thesignals are transferred to one or more floating diffusion nodes at once.Then, an image signal of a corresponding pixel in a row that issequentially selected is output.

A time-of-flight (ToF) camera is a range imaging camera system thatresolves distance based on the speed of light, measuring thetime-of-flight of a light signal between the camera and the subject foreach point of the image. With a time-of-flight camera, the entire sceneis captured with each laser or light pulse. Time-of-flight cameraproducts have become popular as the semiconductor devices have becomefast enough to support such applications. Direct Time-of-Flight imagingsystems measure the direct time-of-flight required for a single laserpulse to leave the camera and reflect back onto the focal plane array.The 3D images can capture complete spatial and temporal data, recordingfull 3D scenes with a single laser pulse. This allows rapid acquisitionand real-time processing of scene information, leading to a wide rangeof applications. These applications include automotive applications,human-machine interfaces and gaming, measurement and machine vision,industrial and surveillance measurements, and robotics, etc.

However, conventional CMOS pixel sensors tend to have drawbacks such aslarge size and high power consumption. Therefore, an improved pixelcircuit and method is desirable for a variety of mobile computer visionapplications.

SUMMARY OF THE INVENTION

In embodiments of the present invention, a ToF depth measurement can becarried out by illuminating the object or scene with light pulses usinga sequence of temporal windows and applying a convolution process to theoptical signal received at the sensor. Embodiments of the inventionprovide an improved pixel circuit and a method that has fasttime-of-flight gating using a single control line per pixel. The pixelcircuit can be implemented based on an improved global shutter CMOSimage sensor process flow. In some embodiments, a shuttering mechanismbased on static gate integration combined with dynamic LDM (lateraldraining modulation) can be realized using either a global reset gate ordrain voltage modulation. Fast signal transfer is enabled in the pixelcircuit using low capacitance and low resistance device structure andinterconnect lines.

According to some embodiments of the present invention, a method isprovided for operating a pixel circuit for time-of-flight (ToF) distancemeasurement. The pixel circuit includes a photodiode, a drain regionadjacent to the photodiode and coupled to a bias voltage, and a shuttergate disposed between the photodiode and the drain region. The shuttergate is controlled by a global shutter signal to apply the bias voltageto bias the photodiode for light sensing. The pixel circuit also has astorage diode and a floating diffusion region. The storage diode iscoupled to the photo diode through a first transfer gate controlled by afirst transfer signal. The floating diffusion region is coupled to thestorage diode through a second transfer gate controlled by a secondtransfer signal. The method includes an exposure period and a samplingperiod. In the exposure period, the method includes activating the firsttransfer gate, using the first transfer signal to couple the photodiodeand the storage diode, activating the photodiode, in a first pluralityof time windows, to sense light reflected from a target as a result of acorresponding plurality of emitted light pulses, wherein a delay timebetween each time window and a corresponding emitted light pulse isdesignated as D1. In the sampling period, the method includes activatingthe second transfer gate, using the second transfer signal, to transfercharges from the storage diode to the floating diffusion region. Thecharges in the floating diffusion region are then sampled to determine asampled signal S1 representing charges collected during the exposureperiod.

According to some embodiments of the present invention, the method caninclude two exposure and sampling phases. In the first exposure andsampling phase, the photodiode is exposed in a first plurality of timewindows in a first exposure period to sense light reflected from atarget as a result of a corresponding plurality of emitted light pulses,wherein a delay time between each time window and a correspondingemitted light pulse is designated as D1. In the first sampling period,the charges are sampled to determine a first sampled signal S1. In thesecond exposure and sampling phase, the photodiode is exposed in asecond plurality of time windows in a second exposure period to senselight reflected from a target as a result of a corresponding pluralityof emitted light pulses, wherein a delay time between each time windowand a corresponding emitted light pulse is designated as D2. In thesecond sampling period, the charges are sampled to determine a secondsampled signal S2. The method further includes determining a distance tothe target based on the first sampled signal S1 and the second sampledsignal S2.

According to some embodiments of the present invention, an image sensordevice includes a plurality of pixel cells arranged in a matrix in apixel array, control circuit for controlling an exposure phase and asampling phase of the image sensor device, and a switching circuit forcoupling a pixel power supply line to a first voltage in an exposurephase and to a second voltage in a sampling phase, the first voltagebeing higher than the second voltage. Each of the plurality of pixelcells includes a photodiode in a semiconductor substrate. A first end ofthe photodiode is coupled to a bias voltage through a shutter gatecontrolled by a global shutter signal. A ground contact couples a secondend of the photodiode to an electrical ground through an electricalground conductive line. Each pixel cell also has a storage diode in thesemiconductor substrate and coupled to a second end of the photodiodephoto diode through a first transfer gate controlled by a first transfersignal. The pixel cell also has a floating diffusion region in thesemiconductor substrate and coupled to the storage diode through asecond transfer gate controlled by a second transfer signal.

In some embodiments of the above image sensor device, the controlcircuit is configured to activate the photodiode in a plurality of timewindows to sense light reflected from a target as a result of acorresponding plurality of emitted light pulses, with a pre-determineddelay time between each time window and a corresponding emitted lightpulse. The photodiode can be activated using a plurality of bias voltagepulses or a plurality of global shutter signal pulses.

According to some embodiments of the present invention, afour-photodiode pixel cell for differential ToF mode operation includesfour photodiodes and four storage diodes. Each storage diode is disposedbetween a first adjacent photodiode and a second adjacent photodiode,and each storage diode is configured to receive photo charges fromeither or both of the first adjacent photodiode and the second adjacentphotodiode. As used herein, the term “photo charges” refers to thecharges generated when light shines on a photodiode. In some cases, theterm “photo charges” is used interchangeably with the term“photoelectrons.” Each photodiode is disposed between a first adjacentstorage diode and a second adjacent storage diode, and each photodiodeis configured to transfer photo charges to either or both of the firstadjacent storage diode and the second adjacent storage diode.

In some embodiments of the above pixel cell, the four photodiodes arearranged in a 2-by-2 array, with each storage diode disposed between twoadjacent photodiodes.

In some embodiments, the pixel cell also has a transfer gate betweeneach pair of adjacent photodiode and storage diode. In some embodiments,the pixel cell also has a charge control gate overlying each storagediode.

In some embodiments, the pixel cell also has four floating diffusionregions, each floating diffusion region disposed adjacent to acorresponding storage diode. The pixel cell also has a transfer gatebetween each pair of adjacent storage diode and floating diffusionregion. In some embodiments, the pixel cell also has a global shuttercontrol gate associated with each photodiode for draining the charges inthe photodiode.

Some embodiments provide an image sensor device including a plurality ofpixel cells, described above, arranged in a pixel array.

According to some embodiments of the present invention, a method foroperating a pixel cell includes exposing the pixel cell to light duringan exposure time window, the pixel cell including four photodiodes andfour storage diodes. Each storage diode is disposed between two adjacentphotodiodes, and each storage diode is configured to receive photocharges from either or both of the two adjacent photodiodes. Eachphotodiode is disposed between two adjacent storage diodes, and eachphotodiode is configured to transfer photo charges to either or both ofthe two adjacent storage diodes.

In some embodiments, for a differential ToF mode, the above method canalso include, during a first time period, transferring collected photocharges from a first pair of photodiodes to a first storage diodedisposed between the first pair of photodiodes, and transferringcollected photo charges from a second pair of photodiodes to a secondstorage diode disposed between the second pair of photodiodes. Themethod includes, during a second time period, transferring collectedphoto charges from a third pair of photodiodes to a third storage diodedisposed between the third pair of photodiodes, and transferringcollected photo charges from a fourth pair of photodiodes to a fourthstorage diode disposed between the fourth pair of photodiodes. Further,for producing a differential signal, the method includes providing a sumof the photo charges from the first storage diode and the second storagediode to a first input of a differential amplifier, and providing a sumof the photo charges from the third storage diode and the fourth storagediode to a second input of the differential amplifier.

In some embodiments of the above method, the method also includestransferring photo charges from the first storage diode and the secondstorage diode to a first floating diffusion region, transferring photocharges from the first storage diode and the second storage diode to asecond floating diffusion region, transferring photo charges from thefirst floating diffusion region to a first sample-and-hold capacitor,and transferring photo charges from the second floating diffusion regionto a second sample-and-hold capacitor. The method also includestransferring signals from the first and second sample-and-holdcapacitors to the differential amplifier.

In some embodiments of the above method, the first pair of photodiodesand the second pair of photodiodes have no photodiode in common, and thethird pair of photodiodes and the fourth pair of photodiodes have nophotodiode in common.

In some embodiments of the above method, for a binning operation, themethod includes transferring collected photo charges in a first pair ofadjacent photodiodes to a first storage diode, and transferringcollected photo charges in a second pair of adjacent photodiodes to asecond storage diode. The method also includes sensing photo charges inthe first storage diode and the second storage diode to provide twosensed signals for binning.

In some embodiments of the above method, for a binning operation, themethod includes transferring collected photo charges in each photodiodeto a respective adjacent storage diode, and sensing photo charges ineach storage diode to provide sensed signals for four sub-pixels.

According to some embodiments of the invention, a pixel cell can includea plurality of photodiodes and a corresponding plurality of storagediodes. Each storage diode is disposed between a first adjacentphotodiode and a second adjacent photodiode, and each storage diode isconfigured to receive photo charges from either or both of the firstadjacent photodiode and the second adjacent photodiode. Each photodiodeis disposed between a first adjacent storage diode and a second adjacentstorage diode, and each photodiode is configured to transfer photocharges to either or both of the first adjacent storage diode and thesecond adjacent storage diode.

In some embodiments, the above pixel cell can also include acorresponding plurality of floating diffusion regions, each floatingdiffusion region disposed adjacent to a corresponding storage diode. Insome embodiments, the pixel cell can also include a transfer gatebetween each pair of adjacent storage diode and floating diffusionregion.

According to some embodiments of the invention, an image sensing devicecan include a plurality of pixel cells arranged in a pixel array, eachpixel cell including four photodiodes, four storage diodes, and fourfloating diffusion regions. Each storage diode is disposed between afirst adjacent photodiode and a second adjacent photodiode, and eachstorage diode is configured to receive photo charges from either or bothof the first adjacent photodiode and the second adjacent photodiode.Each photodiode is disposed between a first adjacent storage diode and asecond adjacent storage diode, and each photodiode is configured totransfer photo charges to either or both of the first adjacent storagediode and the second adjacent storage diode. Each floating diffusionregion is disposed adjacent to a corresponding storage diode. The imagesensing device can include a first summing device for receiving photocharges from a second floating diffusion region and a fourth floatingdiffusion region, and a second summing device for receiving photocharges from a first floating diffusion region and a third floatingdiffusion region. The image sensing device can also include adifferential amplifier coupled to the first and second summing devices,and a control circuit for controlling charge transfer in the imagesensing device.

In some embodiments of the above image sensing device, for adifferential ToF mode, the control circuit is configured for exposing apixel cell to a light during an exposure time window, and, during afirst time period, transferring collected photo charges from a firstpair of photodiodes to a first storage diode disposed between the firstpair of photodiodes and transferring collected photo charges from asecond pair of photodiodes to a second storage diode disposed betweenthe second pair of photodiodes. During a second time period, the controlcircuit is configured for transferring collected photo charges from athird pair of photodiodes to a third storage diode disposed between thethird pair of photodiodes, and transferring collected photo charges froma fourth pair of photodiodes to a fourth storage diode disposed betweenthe fourth pair of photodiodes. Further, the control circuit isconfigured for producing a differential signal by providing a sum of thephoto charges from the second storage diode and the fourth storage diodeto a first input of the differential amplifier, and providing a sum ofthe photo charges from the first storage diode and the third storagediode to a second input of the differential amplifier.

In some embodiments of the above image sensing device, for a binningmode, the control circuit is configured for exposing a pixel cell to alight during an exposure time window, transferring collected photocharges in a first pair of adjacent photodiodes to a first storagediode, transferring collected photo charges in a second pair of adjacentphotodiodes to a second storage diode, and sensing photo charges in thefirst storage diode and the second storage diode to provide two sensedsignals for binning.

In some embodiments of the above image sensing device, for a fullresolution mode, the control circuit is configured for exposing a pixelcell to a light during an exposure time window, transferring collectedphoto charges in each photodiode to a corresponding adjacent storagediode, and sensing photo charges in each storage diode to provide sensedsignals for four sub-pixels.

In some embodiments of the present invention, an image sensing devicecan include an array of photodiodes. The image sensing device alsoincludes a first storage diode disposed between a first pair ofphotodiodes in the array, and the first storage diode is configured toreceive photo charges from each photodiode in the first pair ofphotodiodes. The image sensing device also includes a first floatingdiffusion region disposed adjacent to the first storage diode. Further,the image sensing device also includes a second storage diode disposedbetween a second pair of photodiodes in the array, wherein the secondstorage diode is configured to receive photo charges from eachphotodiode in the second pair of photodiodes. The image sensing devicealso includes a second floating diffusion region disposed adjacent tothe second storage diode, and circuitry configured to receive photocharges from the first floating diffusion region and the second floatingdiffusion region.

In some embodiments of the image sensing device, the circuitry isconfigured to sense photo charges in the first storage diode and thesecond storage diode to provide two sensed signals for binning.

In some embodiments of the image sensing device, the circuitry isconfigured to sum photo charges in the first storage diode and thesecond storage diode.

In some embodiments, the image sensing device can also have a thirdstorage diode disposed between a third pair of photodiodes in the array,and the third storage diode is configured to receive photo charges fromeach photodiode in the third pair of photodiodes. The image sensingdevice can also have a third floating diffusion region disposed adjacentto the first storage diode. A fourth storage diode is disposed between afourth pair of photodiodes in the array, and the fourth storage diode isconfigured to receive photo charges from each photodiode in the fourthpair of photodiodes. A fourth floating diffusion region is disposedadjacent to the fourth storage diode. The circuitry is furtherconfigured to receive photo charges from the third floating diffusionregion and the fourth floating diffusion region. In some embodiments,the circuitry is further configured to sum photo charges in the firststorage diode and the second storage diode, sum photo charges in thethird storage diode and the fourth storage diode, and compare the sum ofphoto charges in the first storage diode and the second storage diodewith the sum of photo charges in the third storage diode and the fourthstorage diode. In some embodiments, the first pair of photodiodes andthe second pair of photodiodes have no photodiode in common, and thethird pair of photodiodes and the fourth pair of photodiodes have nophotodiode in common.

According to some embodiments of the invention, an image sensing deviceincludes a plurality of photodiodes and a plurality of storage diodes.Each storage diode is disposed between a first adjacent photodiode and asecond adjacent photodiode. The image sensing device also includes acontrol circuitry electrically coupling each photodiode to a firstadjacent storage diode and a second adjacent storage diode. The controlcircuitry is configured for alternating between (i) transferring photocharges from each photodiode to the first adjacent storage diode, and(ii) transferring photo charges from each photodiode to the secondadjacent storage diode.

The following description, together with the accompanying drawings, willprovide further understanding of the nature and advantages of theclaimed invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a time-of-flight (ToF) imaging systemfor depth measurement according to an embodiment of the presentinvention;

FIGS. 2A and 2B are diagrams illustrating examples of an image sensorpixel array in a time-of-flight (ToF) imaging system having fiber opticfeedback for calibration according to an embodiment of the presentinvention;

FIG. 3 is a diagram illustrating optical feedback paths in atime-of-flight (ToF) imaging system according to an embodiment of thepresent invention;

FIGS. 4A and 4B are diagrams illustrating a time-of-flight (ToF) imagingsystem with fiber optic feedback for calibration according to anembodiment of the present invention;

FIG. 5 is a timing diagram illustrating a method for time-of-flight(ToF) depth measurement according to an embodiment of the presentinvention;

FIG. 6 is a diagram illustrating a sensed signal versus light to shutterdelay time according to an embodiment of the present invention;

FIG. 7A is a diagram illustrating sensed signals versus light to shutterdelay times of two signals with two shutters according to an embodimentof the present invention;

FIG. 7B is a diagram illustrating simulated signals versus light toshutter delay times of two signals with two shutters according to anembodiment of the present invention;

FIG. 7C is a diagram illustrating simulated signals versus depth for twosignals with two shutters according to an embodiment of the presentinvention;

FIG. 8 is a timing diagram illustrating a method for calibration anddepth measurement in a time-of-flight (ToF) imaging system according toan embodiment of the present invention;

FIG. 9 is another timing diagram illustrating a method for calibrationand depth measurement in a time-of-flight (ToF) imaging system accordingto an embodiment of the present invention;

FIG. 10 is a flowchart illustrating a method for calibration and depthmeasurement in a time-of-flight (ToF) imaging system according to anembodiment of the present invention;

FIG. 11 shows a schematic diagram illustrating a pixel circuit for aglobal shutter image sensor and a cross-sectional diagram for a portionof a pixel including a photodiode according some embodiments of thepresent invention;

FIG. 12 is a waveform timing diagram illustrating a method for operatinga pixel circuit for time-of-flight (ToF) distance measurement accordingto some embodiments of the present invention;

FIG. 13 is a waveform timing diagram illustrating another method foroperating a pixel circuit for time-of-flight (ToF) distance measurementaccording to some embodiments of the present invention;

FIG. 14 is a flowchart that illustrates a method of operating a pixelcircuit for time-of-flight (ToF) distance measurement according to someembodiments of the present invention;

FIG. 15 is a flowchart that illustrates a method of operating a pixelcircuit for time-of-flight (ToF) distance measurement according to someembodiments of the present invention;

FIG. 16 is a schematic diagram illustrating another example of a pixelcell according to embodiments of the present invention;

FIG. 17 is a schematic diagram illustrating another example of a portionof a plurality of pixel cells arranged in a matrix in a pixel arrayaccording to embodiments of the present invention;

FIG. 18 shows a cross-sectional diagram illustrating a pixel cell devicestructure according to some embodiments of the present invention;

FIGS. 19A-19F are top-view diagrams illustrating layout options ofvarious components in a pixel cell according to some embodiments of thepresent invention;

FIGS. 20A-20E illustrate simulation results of pixel cells according tosome embodiments of the present invention;

FIGS. 21A-21C illustrate interconnect layout structures for a pixelarray according to some embodiments of the present invention;

FIGS. 22A-22E illustrate lens layout structures for a pixel cell arrayaccording to some embodiments of the present invention;

FIG. 23 is a block diagram illustrating an image sensing systemaccording to some embodiments of the present invention;

FIG. 24 is a graph of experimental results showing pixel signal versuslaser to shutter delay time according to some embodiments of the presentinvention;

FIG. 25 is a waveform timing diagram illustrating a method for operatinga pixel circuit for global shutter image sensing according to someembodiments of the present invention;

FIG. 26 is a waveform timing diagram illustrating a method for operatinga pixel circuit for rolling shutter image sensing according to someembodiments of the present invention;

FIG. 27 is a simplified top view diagram illustrating a pixel cellaccording to some embodiments of the present invention;

FIG. 28 is a simplified top view schematic diagram illustrating adifferential pixel cell according to some embodiments of the presentinvention;

FIG. 29A is a simplified circuit diagram illustrating the differentialpixel cell according to some embodiments of the present invention;

FIG. 29B is a simplified schematic diagram illustrating a supportcircuit for in-pixel differential mode operation of the pixel circuit ofFIG. 29A according to some embodiments of the invention;

FIG. 30A is a timing diagram illustrating time-of-flight (ToF)operations for the pixel cell of FIGS. 28, 29A, and 29B according tosome embodiments of the invention;

FIG. 30B is a timing diagram illustrating time-of-flight (ToF)operations for the pixel cell of FIG. 11 according to some embodimentsof the invention;

FIG. 30C is a timing diagram illustrating a conventional phasemodulation time-of-flight (ToF) operation;

FIG. 31 is a plot of electrical potentials illustrating the operation ofa photodiode in the pixel cell of FIG. 28 according to some embodimentsof the invention;

FIGS. 32A and 32B are plots of electrical potentials illustrating theoperation of a photodiode in the pixel cell of FIG. 11 according to someembodiments of the invention;

FIG. 33 is a flowchart summarizing a method for a differential ToF modeoperation;

FIG. 34 is a simplified top view schematic diagram illustrating a pixelcell for a binning mode operation according to some embodiments of theinvention;

FIG. 35 is a simplified top view schematic diagram illustrating a pixelcell for a full resolution mode operation according to some embodimentsof the invention;

FIG. 36 is line drawing plot illustrating the layout of a portion of thepixel cell of FIG. 28 according to some embodiments of the invention;and

FIGS. 37A, 37B, and 37C are simplified timing diagrams illustratingdifferent modes of operation that can be implemented using the pixelcell of FIGS. 28-29B according to some embodiments of the invention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Embodiments of the present invention provide a system and method thatenable ToF depth measurement with calibration to provide high accuracyusing optical feedback and fast image processing. A range of depthmeasurements can be calibrated for each frame with minimal effect onsensor performance and power consumption.

The description below is presented with reference to a series of drawingfigures enumerated above. These diagrams are merely examples, and shouldnot unduly limit the scope of the claims herein. In connection with thevarious aspects illustrated and described, one of ordinary skill in theart would recognize other variations, modifications, and alternatives.

FIG. 1 is a diagram illustrating a time-of-flight (ToF) imaging systemfor depth measurement according to an embodiment of the presentinvention. As shown in FIG. 1, a time-of-flight (ToF) imaging system100, also referred to as a ToF digital camera, includes an illuminator110 to transmit light pulses 112 to illuminate a target object 120 fordetermining a distance to the target object. Illuminator 110 can includea pulsed illumination unit and optics for emitting the light pulses 112toward the target object. In this example, illuminator 110 is configuredto transmit light to the target object using, for example, a laser lightsource. However, it is understood that other sources of electromagneticradiation can also be used, for example, infra-red light, radiofrequency EM waves, etc. Imaging system 100 also includes an imagesensor 130 having a gated sensor unit including a light-sensitive pixelarray to receive optical signals from the light pulses in the field ofview (FOV) 132 of the sensor lens. The pixel arrays including an activeregion and a feedback region, as explained below in connection withFIGS. 2A and 2B. Imaging system 100 also has an optical feedback device140 for directing a portion of the light from the illuminator 110 to thefeedback region of the pixel array. The optical feedback device 140provides a preset reference depth. The preset reference depth can be afixed ToF length, which can be used to produce a look up table (LUT)that correlates sensed light vs. depth measurement. In some embodiments,the optical feedback device can fold a direct light from theillumination unit into the field of view (FOV) of the lens in the sensorunit. Imaging system 100 further includes a ToF timing generator 150 forproviding light synchronization and shutter synchronization signals tothe illuminator and the image sensor.

In FIG. 1, ToF imaging system 100 is configured to transmit light pulsesto illuminate a target object 120. Imaging system 100 is also configuredto sense, in the feedback region of the pixel array, light from theoptical feedback device 140, using a sequence of shutter windows thatincludes delay times representing a range of depth. The range of depthcan include the entire range of distance that can be determined by theimaging system. Imaging system 100 calibrates time-of-flight (ToF) depthmeasurement reference information based on the sensed light in thefeedback region of the pixel array. Imaging system 100 is furtherconfigured to sense, in the active region of the light-sensitive pixelarray, light reflected from the target object, and to determine thedistance of the target object based on the sensed reflected light andthe calibrated ToF measurement reference information.

FIG. 2A is a simplified diagram illustrating a pixel array that can beused in imaging system 100 according to an embodiment of the presentinvention. As shown, pixel array 200 includes a plurality of pixels 212,and each pixel in the pixel array includes a photo sensitive element(e.g. a photo diode), which converts the incoming light into a current.Fast electronic switches are used as shutters to control the timing ofthe light sensing operation. A time-of-flight (ToF) camera acquiresdepth images by determining the time during which the light travels froma source to an object and to the sensor of the camera. This can be doneby illuminating the object or scene with light pulses using a sequenceof temporal windows and applying a convolution process to the opticalsignal received at the sensor. Further details are described below. Asshown in FIG. 2A, pixel array 200 includes an active region 210 and afeedback region 220. The active region can be used for determining thedistance of a target object, and the feedback region can be used fordepth calibration. The pixel array can also include an isolation region221 separating the feedback region 220 from the active region 210 toreduce interference. The dimension of the isolation region can beselected to prevent the light from the feedback loop to contaminate theimaging signal collected by the objective lens. In some embodiments, forexample, the isolation region can have a width of about 100 μm-200 μm.In some embodiments, feedback region 220 can be located in part of thepixel array that is outside the field of view, e. g. in a corner, or ina less used region of the pixel array. Therefore, the dedicated feedbackregion of the sensor does not incur much overhead. The small feedbackregion can have a limited number of pixels, for example, from a singlepixel to a 10×10 array of pixels, which allows for fast sensing andsignal processing. In some embodiments, a larger feedback region can beused to provide better signal-to-noise ratio (SNR). Averaging the pixelsin a small array can contribute to the accuracy. In some embodiments,both the feedback and active regions are exposed during the calibrationphase, separately. The difference between the two can be used for thecompensation at run time.

FIG. 2B is a simplified diagram illustrating a pixel array that can beused in imaging system 100 according to another embodiment of thepresent invention. As shown in FIG. 2B, pixel array 250 is similar topixel array 200 of FIG. 2A, but can have more than one feedback regions.Pixel array 250 includes an active region 210 and two or more feedbackregions 220. The pixel array can also include an isolation region 221separating each feedback region from the active region. The isolationregion can reduce interference between the feedback region and theactive region. Pixel array 250 can be used in a ToF imaging systemhaving two illumination sources. In some embodiments, an imaging systemcan include more than two illumination sources and correspondingfeedback sensor regions.

FIG. 3 is a simplified schematic diagram illustrating a portion of thetime-of-flight (ToF) imaging system 100 of FIG. 1. FIG. 3 illustratesthat the optical feedback device is configured to prevent light leakagefrom the optical feedback device 140 to the normal pixels in the array.Light inserted in the edge of the FOV can only hit specific pixels inthe pixel array and light having different angle cannot enter the opticsof the sensor.

In some embodiments, the optical feedback device can be configured tofold a direct light from the illumination unit into the field of view(FOV) of the lens in the sensor unit. FIGS. 4A and 4B are simplifieddiagrams illustrating a time-of-flight (ToF) imaging system 400 withfiber optic feedback for calibration according to an embodiment of thepresent invention. FIG. 4A is a top view and FIG. 4B is a sidecross-sectional view of the imaging system. Imaging system 400 includesan illumination unit 410 and a sensor unit 430 disposed on a printedcircuit board (PCB) 401. As shown in FIGS. 4A and 4B, illumination unit410 includes a diode laser source 412, a collimating lens 414, and adiffuser 416 inside an illumination housing 418. Sensor unit 430includes an image sensor 432, a lens 434, and a lens barrel 436 that ismounted on the image sensor with an adhesive 438. Imaging system 400also has an optical fiber 420 to provide the feedback path. In thisembodiment, optical fiber 420 collects certain amount of light from theinterior of the illumination housing (e. g., from parasitic reflectionsinside) and directs it to a corner 442 of a pixel array 440 of imagesensor 432, but outside lens barrel 436. In some embodiments, opaqueadhesive 438 blocks the light from entering the lens barrel. In thisexample, corner region 442 of the pixel array serves as the feedbackregion of the image sensor.

FIG. 5 is a timing diagram illustrating a method for time-of-flight(ToF) depth measurement according to an embodiment of the presentinvention. In FIG. 5, the horizontal axis is the time, and the verticalaxis is the intensity or magnitude of the light signal. Waveform 1represents the light pulse arriving at the sensor, which can bereflected from the target or provided by the feedback optical device.Wave form 2 represents the shutter window. It can be seen that the lightpulse has a width W_(light), and the shutter window has a width ofW_(shutter). Further, there is a time delay between the leading edge ofthe light and the shutter, D_(L→SH). It can be seen that the amount oflight sensed by the sensor varies with the relative delay of the shutterwith respect to the light.

FIG. 6 is a diagram illustrating the magnitude of sensed light signalversus light-to-shutter delay time according to some embodiments of thepresent invention. In FIG. 6, the horizontal axis is thelight-to-shutter delay, D_(L→SH), and the vertical axis is the amount oflight sensed by the sensor. The diagram is divided into several regions,601 to 605. In region 601, the shutter window is far ahead of the lightpulse (to the left) and the shutter is already closed before the lightarrives. In other words, the light-to-shutter delay is negative. Thus,there is no overlap between the shutter and the light. The delayincreases moving to the right of the horizontal axis. At point 611, theshutter starts to overlap with the light. As the delay increases furtherthrough region 602, the overlap between the shutter and the lightcontinues to increase, and more light is sensed, resulting in the risingcurve in region 602. At point 612, the full width of the light starts tooverlap with the shutter window. In region 603, the shutter is fullyopen throughout the duration of the light pulse, and the width of region603 is determined by the width of the shutter opening W_(shutter) minusthe width of the light pulse W_(light). The magnitude of light receivedin this region is marked “Shutter ON signal.” At point 613, the risingedge of the shutter window is aligned with the rising edge of the lightpulse, and the delay D_(L→SH) is zero, as marked by point 617. In region604, the delay D_(L→SH) continues to increase, and the overlap betweenthe shutter window and the light decreases. As a result, the magnitudeof the sensed light decreases in this region, as shown by the decliningcurve. At point 615, the delay is equal to the light width, and theshutter opens as the light pulse ends; as a result, no light is sensed.In region 605, the shutter opens after the light pulse has alreadypassed. No light is sensed in region 605, and the amount of sensed lightin this region is marked “Shutter OFF signal.” Note that in regions 602and 604, the amount of light collected by the sensor varies depending onthe light-to-shutter delay, D_(L→SH). These regions are used in ToFdepth measurement calibration, as explained below.

FIG. 7A is a diagram illustrating sensed light signals versuslight-to-shutter delay times of two signals with two shutters accordingto an embodiment of the present invention. The time-of-flight (ToF)camera acquires depth images by determining the time which light needsfrom a source to an object and reflected back to the camera. This is canbe done by illuminating the object or scene with a light pulse andapplying a convolution of a sequence of windows with varying delay timesto the received optical signal by the sensor. In some embodiments,multiple groups of calibration light pulses are transmitted using asequence of shutter windows that includes delay times representing arange of depth. Each group of light pulses is followed by a readoutoperation. In each readout, the light from the optical feedback deviceare sensed in the feedback region of the pixel array of the sensor. Thereadout data is then analyzed using a convolution process to determineToF depth data. As described above, in regions 602 and 604 of FIG. 6,the amount of light collected at the sensor varies depending on thelight-to-shutter delay, D_(L→SH). Sensed light data similar to that inFIG. 6 can be collected. These regions are used in ToF depth measurementcalibration. As shown in FIG. 7A, two calibration sequences can becarried out to reduce the effect of unknown reflectivity of the targetobject; and the two sequences are denoted S1 and S2. In an embodiment,the difference in light-to-shutter delay, D_(L→SH) for the two sequencesis equal to the width of the shutter window W_(shutter). Under thiscondition, region 604 of sequence S1 and region 602 of sequence S2 canbe aligned in the plot of FIG. 7A and form slices t-1, t-2, . . . , t-k.In each slice, the amount of light collected in S1 and S2, respectively,represent two portions of the reflected light pulse, and the ratio ofS2/S1 is related to a corresponding depth or distance to the targetobject. The region between points A and B in FIG. 7A represents thedepth range that can be determined by this ToF imager. Data of receivedlight can be collected by measuring at multiple points with delaysbetween A and B in front of a target. Using a convolution process, alook up table (LUT) can be constructed that relates the ratio S2/S1 tothe depth or distance to the target. The initial lookup table can beconstructed in the factory calibration process. In a subsequent ToFdepth measurement, two measurements are made with delays from the sameslice of time in FIG. 7A. A ratio of sensed light S2/S1 is determinedbased on sensed data, and the corresponding depth can be determined fromthe look up table.

FIG. 7B is a diagram illustrating simulated signals versuslight-to-shutter delay times of two signals with two shutters accordingto an embodiment of the present invention. The simulation was carriedout with two shutters on a static test with a flat target at 100 cm fromthe camera, scanning a range of the light-to-shutter delays. Similar toFIG. 7A, the shutter signal (or the number of photo-electrons collectedat the sensor) is plotted for two shutters S1 and S2. In this view,depth can be negative. In the horizontal axis of FIG. 7B, the delay isconverted into depth by the following equation:<depth>=<speed of light>/2*(<electronic delay>−<simulation delayvector>)

In some embodiments, the width of the light pulse is 5-10 nsec, and theshutter window width is 5-15 nsec. The range of delays examined isbetween 5-20 nsec. In some embodiments, the light pulse width can bebetween 3 nsec to 20 sec. The width of the shutter can be in the samerange.

FIG. 7C is a diagram illustrating simulated signals versus depth for twosignals with two shutters according to an embodiment of the presentinvention. FIG. 7C shows data as measured on a rail against a wall atdifferent distances (with 1/distance² decay). It can be seen that thereis a correlation between the ratio of S2/S1 and the depth.

From testing data such as those obtained using methods described inFIGS. 5, 6, and 7A-7C, a look up table (LUT) is constructed in thefactory calibration process. In a ToF depth measurement, a ratio ofS2/S1 is determined based on sensed data, and the corresponding depthcan be determined from the lookup table.

As described above, time-of-flight depth measurement systems can besusceptible to variations in process and operating conditions, such astemperature, voltage, and frame rate, etc. In order to mitigate theeffects of variations, embodiments of the invention provide a system andmethod for run-time calibration of ToF depth measurement using anoptical feedback device as described above. The small number of feedbackpixels allows for fast sensing and signal processing, and with thestrong feedback illumination, for example provided by optical fiber, thenumber of sampling pulses can be greatly reduced. The process ofillumination and readout can be carried out in a short time. As aresult, the depth calibration can be carried out at run-time withoutaffecting the frame rate of the camera. The calibration can be carriedout in each frame. Further, the overhead in power consumption ordedicated feedback pixels is small. Isolation between the feedbackregion and active region of the pixel array is provided to minimizeinterference.

FIG. 8 is a timing diagram illustrating a method for depth profilecalibration between frames of time-of-flight depth measurement accordingto an embodiment of the present invention. The method includes astabilization period 810, a calibration period 820, and a measurementperiod 830. In stabilization period 810, thermal stabilizationillumination pulses are emitted, followed by a dummy readout for thermalstabilization of the sensor. In calibration period 820, thetime-of-flight lookup table (LUT) is calibrated. Here, multiple groupsof calibration illumination pulses P-1, P-2, . . . , P-N are emittedusing a sequence of shutter windows that includes delay timesrepresenting a range of depth. Each group of light pulses is followed bya readout operation, R-1, R-2, . . . , R-N, respectively. In eachreadout, the light from the optical feedback device are sensed in thefeedback region of the pixel array of the sensor. The readout data isthen analyzed using a convolution process to determine ToF depth data asdescribe above in connections with FIGS. 5, 6, and 7A-7C. The depth datais then used to calibrate the lookup table.

The measurement period 830 has two steps 831 and 832. In the first step831, a first group of light pulses S1 with a first shutter delay D1 istransmitted to illuminate the target. Because only a small amount oflight can be collected by the sensor within a shutter window, often alarge number, e.g., several thousand, pulses are sent out and gatheredto increase the signal to noise ratio. During the “S1 read” period, thelight reflected from the target is sensed in the active region of thepixels in the sensor. In the second step 832, a second group of lightpulses S2 with a second shutter delay D2 is transmitted to illuminatethe target. During S2 read, the light reflected from the target issensed in the active region of the pixels in the sensor. Next, the ratioof sensed data readouts S2/S1 are used to determine the distance of thetarget object using the calibrated look up table. In some embodiments,S1 and S2 have preset delays that are chosen in the factory calibrationprocess or in the field of application.

FIG. 9 is a timing diagram illustrating that the depth profilecalibration can fit in between frames of time-of-flight depthmeasurement according to an embodiment of the present invention. FIG. 9is similar to FIG. 8, and is used to explain examples of the length oftime each operation takes within a frame of time-of-flight depthmeasurement. In this embodiment, the thermal stabilization pulses take0.15 msec, and the dummy readout for thermal stabilization takes 0.1msec. Therefore, the length of the stabilization period is about 0.25msec. In the look up table (LUT) calibration period 820, 20 steps ofcalibration light pulses and readouts are used, each with a differentlight-to-shutter delay time. In an example, each step includes 30pulses, each having a pulse width of 150 nsec, followed by a read outoperation of 3 μsec. Thus, the calibration period takes about 0.15 msec.In the measurement period 830, the S1 step can include 1.5 msec of lightpulses (e.g., 1,000 pulses of 150 nsec pulses) followed by a 0.5 msecreadout. Similarly, the S2 step can include 2.0 msec of light pulses,followed by a 0.5 msec readout. In this example, the complete operationincluding stabilization, full range depth calibration, and ToF depthmeasurement takes 4.9 msec. The calibration phase takes about 1/300 ofthe total operation. This optical operation is fast enough to fit in aframe rate of 60 or more frames per second (fps).

The embodiments of the invention provide many advantages overconventional methods. For example, the feedback optical device canprovide strong light for calibration. For example, the feedback opticaldevice can include optical fiber. One or more separate feedback regionsin the pixel array are used for sensing the feedback optical signal. Thefeedback regions are configured in unused or less used regions of thepixel array, and is much smaller than the active region of the array.For example, several pixels are sufficient for feedback sensing if thefeedback optical device can provide a strong signal. The small feedbacksensing region enables quick sensing and fast processing of sensed data,allowing fast calibration of the depth range of interest.

FIG. 10 is a simplified flowchart illustrating a method for ToF depthmeasurement including full range depth calibration according to anembodiment of the present invention. The method described above can besummarized in the flowchart of FIG. 10. As shown, method 1000 includestransmitting light pulses to illuminate a target object, at step 1010.Next, at step 1020, light provided from an optical feedback device issensed in a first region of a light-sensitive pixel array. Here, thefirst region is used as the feedback region. The optical feedback devicereceives a portion of the transmitted light pulses. The feedback opticaldevice includes a preset reference depth for ToF depth measure. Thelight from the optical feedback device is sampled using a sequence ofshutter windows that includes delay times representing a range ofdistances. For ToF depth measurement, the method includes sensing, in asecond region of the light-sensitive pixel array, the scene data whichis light reflected from the target object from the transmitted lightpulses, at step 1030. The second region is the active region of thepixel array. The method includes calibrating time-of-flight (ToF) depthmeasurement reference information based on the sensed light in the firstregion of the pixel array, at step 1040. This process is described indetails above in connection with FIGS. 5, 6 and 7A-7C. Note that,depending on the embodiments, steps 1030 and 1040 can be carried out inany order. For example, after the calibration data (1020) and the scenedata (1040) are captured, data calibration can be processed first andthen the scene data is processed. Alternative, both ToF data calibrationand scene data processing can be carried out simultaneously. Next, themethod includes, at step 1050, determining a distance of the targetobject based on the sensed reflected light and the calibrated ToFmeasurement reference information.

In some embodiments, the method can be carried out in a digital cameracharacterized by a preset frame rate. The calibration can fit in asingle frame period of the camera. In an embodiment, the light from theoptical feedback device is sampled using a sequence of shutter windowsthat includes delay times representing a range of distance. Aconvolution process is then used to correlate the measured signals withthe distance. More details of the method described above can be found inU.S. patent application Ser. No. 15/721,640, filed Sep. 29, 2017,entitled, “Real Time Calibration for Time-of-Flight Depth Measurement,”the entire content of which is incorporated herein by reference.

According to some embodiments of the present invention, an image sensingdevice can include a plurality of pixel cells arranged in a matrix in apixel array and a control circuit for controlling an exposure phase anda sampling phase of the image sensor device. FIGS. 2A and 2B illustrateexamples of a plurality of pixel cells arranged in a matrix in a pixelarray. The image sensing device can be used to implement the method fortime-of-flight (ToF) depth profile according to some embodiments of thepresent invention described above in connection to FIGS. 8 and 9. Anexample of a pixel cell that can be used in such an image device isdescribed below in connection to FIGS. 11-13.

FIG. 11 shows a schematic diagram 1100 illustrating a pixel circuit fora global shutter image sensor and a cross-sectional diagram illustratinga portion of a pixel including a photodiode according some embodimentsof the present invention. A pixel circuit 1110 includes a photo diode1114, for example, a pinned photodiode (PPD), a storage diode (SD) 1118,and a floating diffusion (FD) 1119. As used herein, the storage diode(SD) is also referred to as sensing diffusion (SD). Pixel circuit 1110also includes transistors and corresponding control signals. The controlsignals can be provided by the control circuit in the image sensingdevice. For example, a global shutter transistor 1111 is coupled toreceive a global shutter (GS) control signal, a first transfer gatetransistor 1112 receives a first transfer signal (TX1), a secondtransfer gate transistor 1113 receives a second transfer signal (TX2),and a reset transistor 1115 receives a reset signal (RST). A sourcefollower transistor (SF) 1116 and a select transistor 1117 receives aselect (SEL) signal.

The pixel circuit described above can be configured for correlateddouble sampling (CDS) of charges in the floating diffusion region toreduce the readout noise. In correlated double sampling, a referencevoltage of the pixel (i.e., the pixel's voltage after it is reset) isremoved from the signal voltage of the pixel (i.e., the pixel's voltageat the end of integration) at the end of each integration period.

FIG. 11 also illustrates a cross-sectional view of a pixel cell 1150 ina portion of pixel circuit 1110 according to some embodiments of thepresent invention. As shown, pixel cell 1150 includes a substrate 1151,a photodiode (PPD) 1114, and a ground contact for coupling a second endof the photodiode to an electrical ground (VSS) through an electricalground conductive line. Pixel cell 1150 also has a drain region 1153adjacent to the photodiode and coupled to a bias voltage (VAAGS).Further, pixel cell 1150 has a shutter gate 1111 disposed between thephotodiode PPD 1114 and the drain region 1153. The shutter gate 1111 iscontrolled by a global shutter signal (GS) to apply a bias voltage tobias the photodiode (e.g., a pinned photodiode or PPD) for lightsensing. A storage diode (SD) 1118 is coupled to the photo diode PPD1114 through a first transfer gate 1112 controlled by a first transfersignal (TX1).

An example of a photodiode can include a semiconductor diode having anN-type region formed in a P-type well. The P-type well can be formed inan N-type substrate. A pinned photodiode (PPD) has an additional P+pinning layer disposed at the surface of the N-type region that preventsthe interface from being depleted, and stabilizes the photodiodeelectrically.

Circuit block 1130 includes a row current source transistor 1137 biasedby a signal Vbias. Further, a sample and hold circuit includes a firsttransistor 1131 coupled to a first capacitor 1135. The first transistor1131 receives a first sample-and-hold signal (SH/S) for sampling thecharges during a signal sampling time. A second transistor 1132 iscoupled to a second capacitor 1136. The second transistor 1132 receivesa second sample-and-hold signal (SH/R) for sampling the charges during areset time.

FIG. 12 is a waveform timing diagram illustrating a method for operatinga pixel circuit for time-of-flight (ToF) distance measurement accordingto some embodiments of the present invention. The method can beimplemented using pixel circuit 110 described above in connection toFIG. 11 with control signals generated by a control circuit in the imagesensing device. An example of the image sensing device is illustrated inFIG. 23. As shown in FIG. 12, the method includes an exposure period anda sampling period. In the exposure period, the global shutter (GS)signal is turned on in a first plurality of time windows 1210 toactivate the photodiode to sense light reflected from a target as aresult of a corresponding plurality of emitted light pulses. A delaytime between each time window and a corresponding emitted light pulse isdesignated as D1, as described above in connection with FIGS. 8 and 9.During this time, the first transfer signal (TX1) 1220 is raised toactivate the first transfer gate 1112 in FIG. 11. As shown in FIG. 11,this action couples the photodiode (PPD) 1114 and the storage diode (SD)1118.

In the sampling period, the background signal is first sampled, and thenthe image signal is sampled. To sample the background signal, the reset(RST) signal 1230 and the select (SEL) signal 1240 are turned on. Thetwo sample-and-hold control signals SHR 1250 and SHS 1260 are alsoturned on. As a result, the background signal, also referred to as thereset level signal, is transferred to sample-and-hold capacitors 135 and136.

Next, the second transfer signal (TX2) 1270 is turned on to activate thesecond transfer gate 1113 in FIG. 1. This action causes charges from thephotodiode (PPD) and the storage diode (SD) to be transferred to thefloating diffusion region (FD) in FIG. 11. After the charges aretransferred to the floating diffusion (FD), the second transfer signal(TX2) is turned off, at 1272 in FIG. 12. With the SEL and SHS signalsturned on, the charges in the floating diffusion region (FD) are sampledat the sample-and-hold capacitors 135 and 136 in FIG. 11 for correlateddouble sampling. At this point, charges on capacitors 135 and 136 can becompared to determine a first sampled signal S1 representing chargescollected during the first exposure phase. This sampling operationcorresponds to the S1 read period illustrated in FIGS. 8 and 9.

In FIG. 12, VDD is the power supply voltage. In some embodiments, thepower supply voltage VDD is at a higher voltage 1280 during the exposureperiod than it is 1290 during the sampling period. For example, thepower supply voltage VDD can be at 4.5 V (1280) during the exposureperiod, and the transfer signals TX1 and TX2 can be at 3,3 V (1290)during the exposure period. Therefore, bias voltage, VAAGS or VDD, ishigher than the first transfer signal (TX1) and the second transfersignal (TX2).

In some embodiments, the global shutter gate 1111 in FIG. 11 can beturned on to activate the photodiode in a first plurality of timewindows by providing a first global shutter signal (GS) that has aplurality of global shutter pulses to expose the photodiode. In thiscase, the power supply voltage VAAGS (or VDD) is maintained at avoltage, e.g., 4.5 V, and the first global shutter signal (GS) is usedto apply the pulsed signal.

In some alternative embodiments, the global shutter gate can be turnedon to activate the photodiode in a first plurality of time windows byproviding a power supply voltage VAAGS (or VDD) that has a plurality ofpulses to expose the photodiode. In this case, the first global shuttersignal (GS) is maintained at a voltage, e.g., 4.5 V, and the powersupply voltage VAAGS (or VDD) is used to apply the pulsed signal. Thisembodiment is further illustrated in FIG. 13.

FIG. 13 is a waveform timing diagram illustrating another method foroperating a pixel circuit for time-of-flight (ToF) distance measurementaccording to some embodiments of the present invention. FIG. 13 issimilar to FIG. 12, with one difference. In FIG. 13, the global shuttergate (1111 in FIG. 11) receives a plurality of pulsed signals VAAGS, orVDD. In contrast, in FIG. 12, the global shutter gate 1111 receives aplurality of pulsed signals from the global shutter control signal (GS).In some embodiments, signals VAAGS or VDD include high frequency pulsesignals, for example, in the nanoseconds range. In order to use drainmodulation with signals VAAGS or VDD, a low capacitance drain region 153in FIG. 11 is used.

FIG. 14 is a flowchart that illustrates a method of operating a pixelcircuit for time-of-flight (ToF) distance measurement according to someembodiments of the present invention. As shown in FIG. 14, method 1400summarizes a method of using the pixel circuit described above inconnection with FIGS. 11-13 that can be used in the stabilization period810, calibration period 820, or a first phase of measurement period 830as described above in connection with FIG. 8.

The method 1400 includes an exposure period and a sampling period. Inthe exposure period, at 1410, the method includes activating the firsttransfer gate using the first transfer signal (TX1) to couple thephotodiode (PPD) and the storage diode (SD). At 1420, a plurality ofemitted light pulses is transmitted to a target. At 1430, the methodincludes activating the photodiode in a plurality of time windows tosense light reflected from the target as a result of a correspondingplurality of emitted light pulses. A delay time between each time windowand a corresponding emitted light pulse is designated as D1. The shuttersignal pulses (GS or VDD) are aligned to illumination pulses with apreset first delay D1.

In a sampling period, at 1440, the method includes activating the secondtransfer gate, using the second transfer signal (TX2), to transfercharges from the storage diode (SD) to the floating diffusion region(FD). At 1450, the charges in the floating diffusion region are sampledto determine a first sampled signal S1 representing charges collectedduring the first exposure period.

FIG. 15 is a flowchart that illustrates a method of operating a pixelcircuit for time-of-flight (ToF) distance measurement according to someembodiments of the present invention. As shown in FIG. 15, method 1500summarizes a method of using the pixel circuit described above inconnection with FIGS. 11-13 that can be used in the measurement period830 as described above in connection with FIG. 8. The method 1500includes two exposure-and-sampling phases. In FIG. 15, in a firstexposure period, at 1510, the method includes activating the firsttransfer gate, using the first transfer signal (TX1), to couple thephotodiode (PPD) and the storage diode (SD). At 1520, a plurality ofemitted light pulses is transmitted to a target. At 1530, the methodincludes activating the photodiode in a plurality of time windows tosense light reflected from the target as a result of a correspondingplurality of emitted light pulses. A delay time between each time windowand a corresponding emitted light pulse is designated as D1. In a firstsampling period, at 1540, the method includes activating the secondtransfer gate, using the second transfer signal (TX2), to transfercharges from the storage diode (SD) to the floating diffusion region(FD). Charges in the floating diffusion region are sampled to determinea first sampled signal S1 representing charges collected during thefirst exposure period.

The method further includes, in a second exposure period, at 1550,activating the first transfer gate, using the first transfer signal(TX1), to couple the photodiode (PPD) and the storage diode (SD). At1560, a second plurality of emitted light pulses is transmitted to thetarget, and the photodiode is activated in a corresponding plurality oftime windows to sense light reflected from the target as a result of acorresponding second plurality of emitted light pulses. A delay timebetween each time window and a corresponding emitted light pulse isdesignated as D2. At 1570, in a second sampling period, the methodincludes activating the second transfer gate using the second transfersignal (TX2) to transfer charges from the storage diode (SD) to thefloating diffusion region (FD). At 1580, charges in the floatingdiffusion region are sampled to determine a second sampled signal S2representing charges collected during the second exposure period. At1590, the method includes determining a distance to the target based onfirst sampled signal S1 and second sampled signal S2.

More details about the method are described above in connection to FIGS.8 and 9. For example, distance to the target can be determined based onthe ToF method using a lookup table based on the first sampled signal S1and the second sampled signal S2. The method can be based on aconvolution method described above in connection with FIGS. 8 and 9.

In some embodiments of the invention, an image sensor device is providedfor ToF distance measurement. The image sensing device can include aplurality of pixel cells arranged in a matrix in a pixel array and acontrol circuit for controlling an exposure phase and a sampling phaseof the image sensor device. FIGS. 2A and 2B illustrate examples of aplurality of pixel cells arranged in a matrix in a pixel array. Thecontrol circuit can include circuitry such as pulsed illumination unit110, sensor unit 130, and ToF timing generator 150 as illustrated inFIGS. 1, 3, and FIGS. 4A and 4B. An example of one of the plurality ofpixel cells is illustrated in FIG. 11.

FIG. 16 is a schematic diagram illustrating another example of a pixelcell according to embodiments of the present invention. As shown in FIG.16, pixel cell 1600 includes a photodiode (PPD) in the semiconductorsubstrate (not shown), a first end of the photodiode coupled to a biasvoltage (VAAGS) through a shutter gate (SGT) controlled by a globalshutter signal (GS) and a drain region (Drain). A ground contact (G1)couples a second end of the photodiode to an electrical ground (VSS)through an electrical ground conductive line. Pixel cell 1600 also has astorage diode (SD) in the semiconductor substrate and coupled to thephotodiode through a first transfer gate (TG1) controlled by a firsttransfer signal (TX1). A floating diffusion region (FD) in thesemiconductor substrate is coupled to the storage diode (SD) through asecond transfer gate (TG2) controlled by a second transfer signal (TX2).A switching circuit S1 is configured for coupling a pixel power supplyline to a first voltage (V1) in an exposure phase and to a secondvoltage (V2) in a sampling phase. In this case, the first voltage (V1)can be higher than the second voltage (V2). Pixel cell 1600 further hasa source follower (SF) and a select transistor (SEL) for coupling to abit line that is coupled to a sensing circuit. Part of the sensingcircuit is described above in connection to FIG. 11.

In some embodiments of the image sensing device, the control circuit isconfigured to activate the photodiode in a plurality of time windows tosense light reflected from a target as a result of a correspondingplurality of emitted light pulses, with a pre-determined delay timebetween each time window and a corresponding emitted light pulse. Insome embodiments, a plurality of global shutter signal pulses (GS) isused to activate the photodiode in the plurality of time windows tosense reflected light. In other embodiments, a plurality of bias voltagepulses (VGSAA) is used to activate the photodiode in the plurality oftime windows to sense reflected light. In this case, the photodiode iscoupled to a bias voltage (VAAGS) through a drain region in the pixelcell.

FIG. 17 is a schematic diagram illustrating another example of a portionof a plurality of pixel cells arranged in a matrix in a pixel arrayaccording to embodiments of the present invention. Two pixel cells, Cell1 and Cell 2, are shown in FIG. 17. Each of Cell 1 and Cell 2 is similarto pixel cell 1600 described above in connection to FIG. 16. FIG. 17illustrates a double bit-line configuration in odd and even pixels. Itcan be seen that bit line B1 is associated with Cell 1, and bit line B2is associated with Cell 2. A fast readout can be enabled by the doublebit-line configuration for odd/even pixels together with dedicateddesign of the analog readout mechanism, and a fast interface circuit tocollect the image by the controller.

In embodiments of the present invention, ToF measurement can beimplemented using only a single control line per pixel. In someembodiments, a convolution method for ToF distance measurement isdescribed. This ToF method requires light pulses and shutter windowswith different relative delays. For VR and AR applications, the distanceof interest can be relatively short, e.g., the dimension of a room, inthe range of meters, instead of kilometers. Therefore, that methodrequires very narrow light pulses and shutter windows. In order tosample enough light, multiple (e.g., thousands) of emitted light pulsesand narrow shutter windows in each exposure cycle are needed. Therefore,a fast pixel cell and circuit are provided. In some embodiments, thepixel circuit can operate in the nanoseconds range. In contrast,conventional pixel circuits often operate in the microseconds range.

In some embodiments of the invention, fast time-of-flight (ToF) gatingis achieved by using a single control line per pixel that is isolatedfrom the other control lines to maintain low capacitance. The singlecontrol line can be used to control the global shutter (GS) gatedescribed above. The global shutter (GS) gate can be controlled byeither the global shutter signal (GS) or the bias voltage (VAAGS). Incontrast, conventional ToF pixel technologies can include one of the twocases: 1) switching two signals of the pixel simultaneously, whichrequires good signal matching; or 2) switching a substrate signal, whichrequires process complexity to implement.

In order to enable fast switching of the pixel cells, the photodiode(PPD), the global gate (GS), and the first transfer gate (TX1) areintegrated together, as described above in connection to FIG. 11. Thepixel cell and pixel circuit described herein can be used for ToFimaging, as well as global shutter or rolling shutter imaging, withGS-only modulation. In some embodiments, reduction of resistance betweenPPD and VSS can be achieved by adding a dedicated VSS contact per pixel(as shown in dashed line in FIGS. 16 and 17). Parasitic capacitancebetween the GS line and other global lines are reduced by transferringthe global line to a separate metal layer. The photodiode is configuredto reduce PPD Vpin (and FWC) to improve transient time of chargecarriers. The width and length dimensions of the global shutter linesare increased to improve shutter efficiency. Further, as describedabove, during the ToF exposure period, a high voltage is used to improveshutter efficiency. Further details of the device structure and layoutare described below.

FIG. 18 shows a cross-sectional diagram illustrating a pixel cell devicestructure according to some embodiments of the present invention. InFIG. 18, Cell A is a cross-sectional diagram illustrating a pixel cellaccording to some embodiments of the present invention, and Cell B is across-sectional diagram illustrating a conventional pixel cell. It canbe seen that Cell A has a photodiode (PPD) with a junction depth ofabout 8 um, whereas cell B has a junction depth of about 2 um. One wayto form a deep junction is to decrease the doping level of thesubstrate. The deeper junction depth in Cell A can improve quantumefficiency (QE) light sensing in the pixel cell. According to certainimplementations, an epitaxial layer in which the photodiodes arefabricated can have a thickness in the range of 6 to 9 microns, and thedoping density in the epitaxial layer can be between 2E13 and 5E13.

FIGS. 19A-19F are top-view diagrams illustrating layout options ofvarious components in a pixel cell according to some embodiments of thepresent invention. In each of FIGS. 19A-19F, the pixel cell is shown tohave a photodiode (PPD), a transfer gate (TG), and one or two globalshutter gates (GS). The broken line in each figure is a chargewatershed, which is explained further with reference to the devicesimulation results illustrated in FIGS. 20A-20E.

FIG. 19A illustrates a pixel cell that has one global shutter (GS) gatedisposed at one side of the photodiode (PPD) with a transfer gate (TG)disposed at an adjacent side of the global shutter (GS) gate. FIG. 19Billustrates a pixel cell that has one global shutter (GS) gate disposedat one side of the photodiode (PPD) with a transfer gate (TG) disposedat an adjacent side of the global shutter (GS) gate. Further, thetransfer gate (TG) has an offset with respect to a center of the side ofthe photodiode.

FIG. 19C illustrates a pixel cell that has one global shutter (GS) gatedisposed at one side of the photodiode (PPD) with a transfer gate (TG)disposed at an opposite side of the global shutter (GS) gate. FIG. 19Dillustrates a pixel cell that has one global shutter (GS) gate disposedat one side of the photodiode (PPD) with a transfer gate (TG) disposedat an opposite side of the global shutter (GS) gate. Further, thetransfer gate (TG) has an offset with respect to a center of the side ofthe photodiode.

FIG. 19E illustrates a pixel cell that has two global shutter (GS) gatesdisposed at adjacent sides of the photodiode (PPD) with a transfer gate(TG) disposed at another side of the global shutter (GS) gate. FIG. 19Fillustrates a pixel cell that has two global shutter (GS) gates disposedat adjacent sides of the photodiode (PPD) with a transfer gate (TG)disposed at another side of the global shutter (GS) gate. Further,transfer gate (TG) has an offset with respect to a center of the side ofthe photodiode.

In some embodiments, the pixel cell illustrated in FIG. 19B provides thebest pixel device performance. The integration of light during the GS ONstate can contaminate the GS signal. Therefore, it is desirable tominimize the TX1 side that is exposed to the imaging system.

FIGS. 20A-20E illustrate simulation results of pixel cells according tosome embodiments of the present invention. FIG. 20A shows a perspectiveview of a pixel cell illustrating a photodiode (PPD) with a globalshutter gate (GS) and a transfer gate (TX) disposed at an adjacent sideof the PPD. FIG. 20A also shows equipotential electrostatic potentialinside the PPD of a pixel cell under a shutter open condition withintegration on and a shutter in closed position with integration turnedoff. When the shutter is in open condition with integration on, all thePPD area is integrated. When the shutter is in closed condition withintegration off, only a small portion of the PPD area is integrated. Abroken line marks a charge watershed. FIG. 20B illustrates additionalsimulation results for the device illustrated in FIG. 20A.

FIG. 20C shows a perspective view of a pixel cell illustrating aphotodiode (PPD) with a global shutter gate (GS) and a transfer gate(TX) disposed on opposite sides of the PPD. FIG. 20C also showsequipotential electrostatic potential inside the PPD of a pixel cellunder a shutter open condition and a shutter closed position. A chargewatershed is marked by a broken line. FIG. 20D illustrates additionalsimulation results for the device illustrated in FIG. 20C.

FIG. 20E shows simulated equipotential electrostatic potential contoursin a pixel cell having a photodiode (PPD), a transfer gate (TX), and twoglobal shutter gates (GS), which can be referred to as a Double GS.

FIGS. 21A-21C illustrate interconnect layout structures for a pixelarray according to some embodiments of the present invention. FIG. 21Ashows a pinned photo diode (PPD) with various control signal lines andpower supply lines implemented in different metal levels. For example,control signal lines for RST, SEL, TX1, and TX2 are implemented in thefirst level metal M1. Voltage supplies VDDA and VSSA, as well as theoutput signal line PixelOUT are implemented in the second level metalM2. The global shutter signal (GS) line is implemented in a third levelmetal M3. It is noted that the global shutter signal (GS) line is a widemetal line in M3, which can reduce line resistance and capacitance withadjacent lines. FIG. 21A also identifies various cross-sectional cutlines, AA′, BB′, CC′, and DD′ for further illustration below.

FIG. 21B illustrates cross-sectional views along cut lines of the pixelcell structure of FIG. 21A. It can be seen from cross-sectional diagramsalong cut line AA′ that BB′ that the spacing between M3 and M1 is large,and overlaps between M2 and M3 and M1 are small. As a result, thecapacitance between various metal lines can be reduced.

FIG. 21C illustrates cross-sectional views along cut lines of the pixelcell structure of FIG. 21A. It can be seen from cross-sectional diagramsalong cut line CC′ that DD′ that the metal layers M1, M2, and M3 aredisposed offset from the photodiode (PPD). This layout can provide largeimage sensing areas in the photodiode.

FIGS. 22A-22E illustrate lens layout structures for a pixel cell arrayaccording to some embodiments of the present invention. FIGS. 22A-22Cillustrate a top view of a portion of a pixel cell array in which eachpixel cell has a photodiode and the metal interconnect structures asshown in FIGS. 21A-21C. FIG. 22A illustrates an arrangement in which themicro lenses are positioned in the center of the pixel cells over thephotodiode. FIG. 22B illustrates an arrangement in which the microlenses are shifted to the right with respect to the photodiodes in thepixel. FIG. 22C illustrates an arrangement in which the micro lenses areshifted downwards with respect to the photodiodes in the pixel. Eachlens is suitably positioned relative to its associated photodiode inorder to steer a chief ray arriving through an associated externalimaging lens (not shown) to the center of the photodiode, so as fosterimproved light collection efficiency.

FIG. 22D is a cross-sectional view of a portion of the pixel cell arrayalong a cut line E-E′ as shown in FIGS. 22A-22C. FIG. 22E is across-sectional view of a portion of the pixel cell array along a cutline F-F′ as shown in FIGS. 22A-22C. FIGS. 22D and 22E illustrate theadvantage of having wide micro lenses in a pixel array with lightentrance into the pixel at the top of the pixel array for a normal caseand an extreme case. It can be seen in FIGS. 22D and 22E that, becausethe photo diode is wide, it can collect photo electrodes from a widelight entrance range.

FIG. 23 is a block diagram illustrating an image sensing system forevaluating the methods described above according to some embodiments ofthe present invention. As shown in FIG. 23, an image sensing device orsystem 2300 can include a plurality of pixel cells arranged in a matrixin a pixel array 2310 and a control circuit or controller 2320 with fastglobal shutter circuitry for controlling an exposure phase and asampling phase of the image sensor device. Image sensing system 2300 canalso include circuits for controlling the illumination timing such as adelay generator 2330, a short pulse laser generator 2340, a collimator2350, and a diffuser 2360. Image sensing system 2300 can also have aprocessor 2370 and a power supply system 2380. It should be understoodthat this system is used for evaluating the methods described above. Inactual use, light emitted by the laser generator 2340 and collimated bythe collimator 2350 is reflected from objects in the environment beforereaching the pixel array 2310. An imaging lens (not shown) can bedisposed in front of the pixel array 2310.

An example of the pixel cell that can be used in such an image device isdescribed above in connection to FIGS. 11-13. The image sensing devicecan be used to implement the method for a time-of-flight (ToF) depthprofile according to some embodiments of the present invention describedabove in connection to FIGS. 11-22E.

In some embodiments, the ToF pixel circuit of FIG. 11, which includesthe circuitry for six transistors (6T), can have a cell pitch of 4.5 umin semiconductor fabrication technology node of 110 nm. In some cases, aquantum efficiency (QE) of about 30% has been achieved. In someembodiments, fast time-of-flight gating enables short integration timeas small as 5 nsec. Certain implementations of the invention includepixels with six transistors and are characterized by a ratio of the cellpitch to semiconductor node used to in fabrication of less than 50, aquantum efficiency greater than 25% and integration times less than 7nanoseconds. Different array voltages can be used during global resetand during pixel array readout to make efficient transfer/draintransitions. Ambient light is integrated only during the short time thatthe sensor is open for integration, typically ˜0.5 msec. The amount ofoperational ambient light depends on the flux of the active illuminationduring the integration time.

Embodiments of the invention can provide fast time-of-flight gating toenable short integration time as small as 5 nsec. High quantumefficiency and a large optical aperture across the entire pixel arraycan save infrared illumination power consumption. The image device andsystem can support mobile computer vision system such as virtualreality, augmented reality, etc., by having a small pixel and low systempower, with passive and active illumination and ToF. The widephoto-diodes and high pixel array resolution are configured to supportVGA or higher computer vision applications. Low noise exists in globalshutter operation as well as rolling shutter operation modes. Ambientlight integration is minimized to enable outdoor operation, e.g., underdirect sunlight.

As an example, capacitance and timing calculation is carried out for aVGA array (640×480) of 4.5 um at 110 nm process node. The pixelcapacitance (including gate+parasitic interconnect) is C_(pixel)=6 f F.The resistance of aluminum interconnect is

$R_{pixel} = {1{\frac{\Omega}{pixel}.}}$The number of pixels (half VGA rows) is n=320, 240 (for H or V drivefrom both sides). The delay of an RC chain from the array side to thecenter of the array can be calculated as follows:

Elmore Delay:

${\tau = {\frac{R_{pixel} \cdot C_{pixel} \cdot n^{2}}{2} = 0.5}},{0.17\mspace{14mu}{nsec}\mspace{14mu}{\left( {H,V} \right).}}$Thus, nanosecond level delays can be achieved in the pixel circuitdescribed above.

FIG. 24 is a graph of experimental results showing a pixel signal versuslaser to shutter delay time according to some embodiments of the presentinvention. The graph is similar to those illustrated in FIG. 6 and FIGS.7A and 7B described above. It can be seen that the rise/fall time isabout 2 nsec modulated at 3.3 V. The pixel signal contrast ratio isabout 20:1. It is noted that the data was measured using a highinductance off-the-shelf package. The performance is expected to beimproved by using a low inductance packaging, such as a chip-on-boardpackage.

The pixel circuit and image sensing device are described above in thecontext of ToF imaging. However, the pixel circuit and image sensingdevice described above can also be used for global shutter (GS) imagingand rolling shutter (RS) imaging, as illustrated below with reference toFIGS. 25 and 26.

FIG. 25 is a waveform timing diagram illustrating a method for operatinga pixel circuit for global shutter image sensing according to someembodiments of the present invention. The method can be implementedusing pixel circuit 110 described above in connection to FIG. 11. In theglobal shutter method, all signals are photo-electrically converted byall photo elements in one frame. The signals are transferred to one ormore floating diffusion nodes at once. As shown in FIG. 25, globalshutter image sensing includes an exposure period and a sampling period.In the exposure period, the global shutter (GS) gate is turned on (2510)along with the first transfer signal (TX1) 2520 to activate thephotodiode to sense light reflected from a target as a result of anemitted light. In the sampling period, the operation is similar to thatdescribed above in connection with FIG. 12.

FIG. 26 is a waveform timing diagram illustrating a method for operatinga pixel circuit for rolling shutter image sensing according to someembodiments of the present invention. The method can be implementedusing pixel circuit 110 described above in connection to FIG. 11. In therolling shutter method, signals are photo-electrically converted byphoto elements in each row in one frame. The signals are transferred toone or more floating diffusion nodes in each row that is sequentiallyselected, and an image signal of a corresponding pixel is output. Asshown in FIG. 26, rolling shutter image sensing can include an exposureperiod and a sampling period. In both the exposure period and thesampling period, the global shutter (GS) gate is maintained at a lateraloverflow potential (2510), and the first transfer signal (TX1) 2620 isat an ON state, to activate the photodiode to sense light reflected froma target as a result of an emitted light.

According to some embodiments of the invention, an alternative pixelcell can include a plurality of photodiodes and a correspondingplurality of storage diodes. For example, in some embodiments, eachpixel cell includes four photodiodes, four storage diodes, and fourfloating diffusion regions. In other embodiments, a different number ofthese components can be utilized. In some embodiments, each storagediode is disposed between a first adjacent photodiode and a secondadjacent photodiode, and each storage diode is configured to receivephoto charges from either or both of the first adjacent photodiodeand/or the second adjacent photodiode. Further, each photodiode isdisposed between a first adjacent storage diode and a second adjacentstorage diode, and each photodiode is configured to transfer photocharges to either or both of the first adjacent storage diode and/or thesecond adjacent storage diode. Further, each floating diffusion regionis disposed adjacent to a corresponding storage diode.

In some embodiments, the pixel cell can include an image sensor devicethat can perform in-pixel differential ToF determination. In theseembodiments, the light reflected from the target is effectivelycollected, and most of the ambient light is not collected. The in-pixeldifferential mode signals are used for ToF determination and can furthercancel the received ambient light. In some embodiments, the pixel cellcan be used in a binning mode, in which multiple photodiodes can begrouped into bins for light collection. Alternatively, the pixel cellcan also be used in a full resolution mode, in which each photodiode canoperate as a sub-pixel. These and other modes of operation are describedin further detail below. In some embodiments, an image sensor device maybe configured to switch between two or more of the modes of operationdescribed herein. For example, in some embodiments, an image sensor mayswitch or be instructed to switch from one mode of operation to anotherbased on user input, sensor data, a predetermined schedule ofoperations, and the like. In some implementations, an image sensordevice may not be configured to switch between two or more of the modesof operation described herein, but may instead be configured to functionin accordance with a single one of the modes of operation describedherein.

FIG. 27 is a simplified top view diagram illustrating a pixel cellaccording to some embodiments of the present invention. As shown in FIG.27, pixel cell 2700 includes a photodiode 2710, a storage diode, alsoreferred to as a storage diffusion 2720, and a readout circuit 2730. Inan application, pixel cell 2700 can be used to implement the pixel celland pixel circuit illustrated above as 1100 in FIG. 11. For example,photodiode 2710 in FIG. 27 corresponds to the pinned photodiode (PPD)1114 in FIG. 11, storage diode 2720 corresponds to storage diode (SD)1118 in FIG. 11, and readout circuit 2730 correspond to part of thepixel circuit 1110 and the circuit block 1130 in FIG. 11. The term“storage diode” is used interchangeably with “sensing diffusion” and“sensing diode.”

Further, in FIG. 27, charge transfer from photodiode 2710 to storagediode 2720 is controlled by a first transfer signal TX1 on a firsttransfer gate 2740 corresponding to first transfer gate 1112 in FIG. 11.Charge transfer from storage diode 2720 to readout circuit 2730 iscontrolled by a second transfer signal TX2 on a second transfer gate2750 corresponding to second transfer gate 1113 in FIG. 11.

FIG. 28 is a simplified top view schematic diagram illustrating adifferential pixel cell according to some embodiments of the presentinvention. As shown in FIG. 28, pixel cell 2800 includes fourphotodiodes: a first photodiode PPD1, a second photodiode PPD2, a thirdphotodiode PPD3, and a fourth photodiode PPD4. Pixel cell 2800 alsoincludes four storage diodes: a first storage diode SD1, a secondstorage diode SD2, a third storage diode SD3, and a fourth storage diodeSD4.

Pixel cell 2800 also include four readout circuits, a first read outcircuit RO_1, a second read out circuit RO_2, a third read out circuitRO_3, and a fourth read out circuit RO_4. Each readout circuit includesa floating diffusion region disposed adjacent to a corresponding storagediode. For example, first read out circuit RO_1 includes a firstfloating diffusion region FD1 disposed adjacent to first storage diodeSD1, second read out circuit RO_2 includes a second floating diffusionregion FD2 disposed adjacent to second storage diode SD2, third read outcircuit RO_3 includes a third floating diffusion region FD3 disposedadjacent to third storage diode SD3, and fourth read out circuit RO_4includes a fourth floating diffusion region FD4 disposed adjacent tofourth storage diode SD4. The floating diffusion regions are chargestorage regions for holding photo charges to be transferred to a readout circuit.

Moreover, FIG. 28 shows two charge summing devices, AMP1 and AMP2, whichare configured to integrate photo charges from two read out circuits.Charge summing devices can be implemented using sample-and-holdcapacitors, summing amplifiers, or the like. In these embodiments,charge summing devices AMP1 and AMP2 are not included in pixel cell 2800as illustrated by their position outside the dashed line representingpixel cell 2800. The summation of charges on PPD1/PPD4 to PPD2/PPD3 canbe performed outside the pixel array, for example, on thesample-and-hold (S/H) capacitors, located in the column parallelcircuits, just before converting it to digital utilizing an ADC(Analog-to-Digital converter) circuit.

Similar to pixel cell 2700 illustrated in FIG. 27, in each of thephotodiodes in pixel cell 2800 illustrated in FIG. 28, the transfer ofphoto charges from the photodiodes (PPD) to the storage diodes (SD) iscontrolled by a first transfer gate signal TX1, and the transfer ofphoto charges from the storage diode (SD) to a floating diffusion (FD)in the read out circuit is controlled by a second transfer gatecontrolled by a second transfer signal TX2. However, in FIG. 28, thefirst transfer signal TX1 is split into four signals with differentphases: TX1_1, TX1_2, TX1_3, and TX1_4, and the second transfer signalTX2 is split into two transfer signals with different phases: TX2_1 andTX2_2.

In some embodiments, pixel cell 2800 can also include four chargecontrol gates responsive to a charge control signal TX3, each chargecontrol gate disposed over a storage diode. In FIG. 28, each chargecontrol gate is shown as a dotted rectangle over a corresponding storagediode, SD1, SD2, SD3, and SD4, respectively. In some embodiments, thecharge control gates are controlled by a common storage diode chargecontrol signal TX3. The charge control gates are kept at a constantvoltage by the charge control signal TX3 to maintain photo charges inthe differential ToF mode. TX3 can prevent spillback and reduce thetoggled gate capacitance. TX3 can also maintain the shapes of the wellsand desirable potential slopes for charge storage and transfer. Instandard mode and during readout, TX3 is toggled simultaneously orconcurrently with transfer gates TX1_N, where N=1, 2, 3, or 4, so theyact as a single gate. In some embodiments, there is no implanted dopedregion between TX3 and transfer gates TX1_N.

FIG. 29A is a simplified circuit diagram illustrating the differentialpixel cell according to some embodiments of the present invention. FIG.29A shows a differential pixel cell 2900 for cell (i, j) for column iand row j in a pixel array. As shown in FIG. 29A, pixel cell 2900includes a first photodiode PPD1(i, j), a second photodiode PPD2(i, j),a third photodiode PPD3(i, j), and a fourth photodiode PPD4(i, j). Pixelcell 2900 also includes a first storage diode SD1(i, j), a secondstorage diode SD2(i, j), a third storage diode SD3(i, j), and a fourthstorage diode SD4(i, j). Pixel cell 2900 also include four readoutcircuits, a first read out circuit RO_1, a second read out circuit RO_2,a third read out circuit RO_3, and a fourth read out circuit RO_4. In anapplication, circuitry of pixel cell 2900 can be used to implement oneor more components of pixel cell 2800, as described above with referenceto FIG. 28.

In FIG. 29A, transfer signal TX1_1 controls the transfer of photocharges from the first photodiode PPD1(i, j) to the first storage diodeSD1(i, j) through control gate 2911. Transfer signal TX1_2 controls thetransfer of photo charges from the fourth photodiode PPD4(i, j) to thefirst storage diode SD1(i, j) through control gate 2912. Chargestransfer from the first storage diode SD1(i, j) to the first readoutcircuit RO_1 is controlled by a transfer signal TX2_2 on a transfer gate2922.

Similarly, transfer signal TX1_3 controls the transfer of charges fromthe first photodiode PPD1(i, j) to the second storage diode SD2(i, j)through control gate 2913. Transfer signal TX1_4 controls the transferof charges from the fourth photodiode PPD4(i, j) to the second storagediode SD2(i, j) through control gate 2914. Charges transfer from thesecond storage diode SD1(i, j) to the second readout circuit RO_2 iscontrolled by a transfer signal TX2_1 on a transfer gate 2921.

Further, in FIG. 29A, transfer signal TX1_2 controls the transfer ofcharges from the third photodiode PPD2(i, j) to the third storage diodeSD3(i, j) through control gate 2932. Transfer signal TX1_1 controls thetransfer of charges from the third photodiode PPD3(i, j) to the thirdstorage diode SD3(i, j) through control gate 2931. Charges transfer fromthe third storage diode SD3(i, j) to the third readout circuit RO_3 iscontrolled by a transfer signal TX2_1 on a transfer gate 2941.

Similarly, transfer signal TX1_4 controls the transfer of charges fromthe second photodiode PPD2(i, j) to the second storage diode SD2(i, j)through control gate 2934. Transfer signal TX1_3 controls the transferof charges from the third photodiode PPD3(i, j) to the fourth storagediode SD4(i, j) through control gate 2933. Charges transfer from thethird storage diode SD3(i, j) to the fourth readout circuit RO_4 iscontrolled by a transfer signal TX2_2 on a transfer gate 2942.

The operations of read out circuits RO_1, RO_2, RO_3, and RO_4 aresimilar to the read out circuit described above in connection with FIG.11. Therefore, only a simplified circuit is shown in each of read outcircuits RO_1, RO_2, RO_3, and RO_4, including floating diffusionregions FD1<j>, FD2<j>, FD3<j>, and FD4<j>, four control gates toreceive reset signals RST<j>, four source follower transistors SF1<j>,SF2<j>, SF3<j>, and SF4<j>, and four more control gates to receiveselect signals SEL<j>. The floating diffusion regions in read outcircuits RO_1, RO_2, RO_3, and RO_4 are similar to the floatingdiffusion regions described for receiving charges from the storagedevices with reference to FIG. 28. The source followers (SF) and resetgates (RST) in FIG. 29B perform similar functions as correspondingelements in FIG. 11.

Pixel cell 2900 also includes a global shutter gate for coupling eachphotodiode to a power supply VDD, and a global shutter signal (GS) forcontrolling a global shutter gate. In FIG. 29, to simplify the drawing,only one global shutter gate (2901) is shown associated with photodiodePPD1(i, j) to represent the global shutter gate supporting the globalshutter signal (GS) and power supply VDD. However, it will beappreciated that each of the four photodiodes has a global shutter gateassociated with each photodiode to enable a respective global shuttersignal (GS). In some embodiments, the Global Shutter signal (GS) has twofunctions. The first function is to provide an overflow drain to avoidblooming, and the second is to enable global reset to empty thephotodiodes outside the integration sequence.

In some embodiments, read out circuits RO_1, RO_2, RO_3, and RO_4 can beshared with adjacent pixel cells. Therefore, some circuit components andsignals associated with adjacent pixel cells are shown in FIG. 29A, suchas TX2_1(i−1, j), SD3(i−1, j), TX2_2(i, j−1), SD4(i, j−1), TX2_1(i,j+1), SD2(I, j+1), TX2_2(i+1, j), and SD1(i+1, j). These arrangementscan facilitate simultaneous or concurrent charge transfer from SD2 andSD4, as well as simultaneous or concurrent charge transfer from SD1 andSD3.

FIG. 29B is a simplified schematic diagram illustrating a supportcircuit for in-pixel differential mode operation of the pixel circuit2900 of FIG. 29A according to some embodiments of the invention. Asshown in FIG. 29B, in circuit 2950, summing device AMP1 includessample-and-hold capacitors SH2 and SH4 for receiving output signals fromread out circuits RO_2 and RO_4 in FIG. 29A, respectively. Similarly,summing device AMP2 includes sample-and-hold capacitors SH1 and SH3 forreceiving output signals from read out circuits RO_1 and RO_3 in FIG.29A, respectively. Circuit 2950 also includes a differential amplifier2970 for receiving signals from the sample-and-hold capacitors todetermine differential signals.

FIGS. 30A, 30B, and 30C are timing diagrams illustrating time-of-flight(ToF) operations for three different pixel cells. FIG. 30A is a timingdiagram illustrating a differential pulsed time-of-flight (ToF)operations for the pixel cells of FIGS. 28, 29A, and 29B according tosome embodiments of the invention. In FIG. 30A, curve 3010 represents alight pulse emitted from a light source toward a target for ToFdetermination. Curve 3011 represents an ambient light signal. Theoperational ToF range is defined by the separation between the shortestdistance and the longest distance measurable using the ToF system.Timing curve 3016 represents an exposure time window during which thephotodiodes are enabled to sensing light input. In some embodiments,timing curve 3016 can represent a control signal in a lateral drainarrangement as described in connection with FIGS. 11-13. As illustratedin FIG. 30A, the temporal duration of timing curve 3016, which can bereferred to as a lateral drain, extends from a time prior to the onsetof timing pulse #1 3001 (associated with the shortest distance measuredfrom the target to the pixel cells) to a time following the terminationof timing pulse #4 3004 (associated with the longest distance measuredfrom the target to the pixel cells). That is, the temporal duration oftiming curve 3016 is greater than the time duration associated with theoperational ToF range. Ambient light received outside the operationalwindow associated with the operational ToF range can be drained usingtiming curve 3016 associated with the lateral drain, thereby reducingphoto charges associated with ambient light and improving the signal tonoise ratio.

Timing curves 3012 and 3014 represent timing windows during whichcharges are transferred from the photodiodes to the storage diodes. Thecharges in the storage diodes are then transferred to the floatingdiffusion regions in the readout circuits. Four timing pulses arelabeled 3001, 3002, 3003, and 3004, respectively. During these timingpulses, photons that are collected by the photodiodes can be directed toone of one or more summing devices (e.g., AMP1 or AMP2) as describedmore fully below, depending on the time of arrival of the photons.

The time duration associated with operational ToF range extends from thebeginning of timing pulse #1 3001 through the end of timing pulse #43004. For example, in different ToF frames, the ToF target may moveslowly and monotonically from a short distance to a long distance. Whenlight pulse 3010 is emitted toward the target and is reflected back fromthe target, at first the reflected light is captured by pulse #1, thenby pulses #1 and #2, then by pulse #2, then by pulses #2 and #3, then bypulse #3, then by pulses #3 and #4, then by pulse #4. After that, thereflected light stops being integrated, for example, when the targetmoves outside the operational ToF range.

With reference to FIG. 28, AMP2 (referred to as Storage 2 in the timingchart in FIG. 30A) combines charges from SD1 and SD3 during timingpulses 1 and 3, and AMP1 (referred to as Storage 1 in the timing chartin FIG. 30A) combines charges from SD2 and SD4 during timing pulses #23002 and timing pulse #4 3004. As shown above in FIG. 29B, the outputsignals from AMP1 and AMP2 are coupled to a differential amplifier 2970to produce a differential signal. The differential signal generatedaccording to the timing chart in FIG. 30A can vary with respect to ToFdepth/time shift, when the returning light is integrated between timingpulse #1 (in 3012) to timing pulse #4 (in 3014). Therefore, differentialsignals based on the light reflected from the target are used for ToFdetermination. Further, the differential signal can eliminate or reducecontribution from ambient light.

In a differential mode operation, the charge transfers can be describedwith reference to FIGS. 28, 29A, and 29B. Transfer signals TX1_1 andTX1_2 are toggled together, and transfer signals TX1_3 and TX1_4 aretoggled together in a complementary fashion. For example, during chargesummation of phase 1, as shown by the timing pulse #1 3001 in FIG. 30A,under the control of transfer signals TX1_1 and TX1_2, photo chargesfrom the photodiode PPD1 and the photodiode PPD4 are transferred tostorage diode SD1, and photoelectrons from the photodiode PPD2 and thephotodiode PPD3 are transferred to storage diode SD3. The charges instorage diodes SD1 and SD3 are transferred to floating diffusion regionsin the respective readout circuits, and the output signals from readoutcircuits RO_1 and RO_3 are then transferred to amplifier AMP2, where thecharges can be stored into sample and hold capacitors.

Similarly, during charge summation of phase 2, as shown by the timingpulse #2 3002 in FIG. 30A, under the control of transfer signals TX1_3and TX1_4, photo charges from the photodiode PPD1 and the photodiodePPD2 are transferred to storage diode SD2, and photo charges from thephotodiode PPD3 and the photodiode PPD4 are transferred to storage diodeSD4. The charges in storage diodes SD2 and SD4 are transferred tofloating diffusion regions in the respective readout circuits, and theoutput signals from readout circuits RO_2 and RO_4 are input toamplifier AMP1, where the charges can be stored into sample and holdcapacitors.

Similarly, during timing pulse #3 3003, under the control of transfersignals TX1_1 and TX1_2, photo charges from the photodiode PPD1 and thephotodiode PPD4 are transferred to storage diode SD1, and photo chargesfrom the photodiode PPD2 and the photodiode PPD3 are transferred tostorage diode SD3. The charges in storage diodes SD1 and SD3 aretransferred to floating diffusion regions in the readout circuits. Theoutput signals from readout circuits RO_1 and RO_3 are then transferredto amplifier AMP2, where the charges can be stored into sample and holdcapacitors.

During timing pulse #4 3004, under the control of transfer signals TX1_3and TX1_4, photo charges from the photodiode PPD1 and the photodiodePPD2 are transferred to storage diode SD2, and photo charges from thephotodiode PPD3 and the photodiode PPD4 are transferred to storage diodeSD4. The charges in storage diodes SD2 and SD4 are transferred tofloating diffusion regions in the readout circuits. The output signalsfrom readout circuits RO_2 and RO_4 are input to amplifier AMP1, wherethe charges can be stored into sample and hold capacitors.

In some embodiments, transfer signals TX2_1 and TX2_2 are used totransfer photo charges from the storage diodes to the floating diffusionregions. For example, as shown in FIG. 28, under the control of transfersignal TX2_1, photo charges from storage diode SD2 are transferred tofloating diffusion region FD2, and photo charges from storage diode SD3are transferred to floating diffusion region FD3. Similarly, under thecontrol of transfer signal TX2_2, photo charges from storage diode SD1are transferred to floating diffusion region FD1, and photo charges fromstorage diode SD4 are transferred to floating diffusion region FD4. Insome embodiments, these charge transfers can take place after timingpulse #4 in FIG. 28. In an alternative embodiment, these chargetransfers can take place after timing pulse #3 and then after timingpulse #4 in FIG. 28.

As shown in FIG. 29B, the output signals from amplifiers AMP1 and AMP2are then provided to the differential amplifier 2970, where differentialsignals are formed. Therefore, differential signals based on the lightreflected from the target are used for ToF determination. As explainedabove, the differential signal generated according to the timing chartin FIG. 30A can vary with respect to ToF depth/time shift, because thereturning light is integrated between timing pulse #1 (in 3012) andtiming pulse #4 (in 3014).

As shown in FIG. 30A, substantially all of the light reflected from thetarget can be collected. In this embodiment, a high intensity lightsource is used as the emitter in comparison with the ambient lightsignal 3011. As a result, in some embodiments, a small amount of ambientlight is collected in comparison with reflected light. Light pulse 3010is transmitted to the target and returns. The time range between thestart of timing pulse #1 of 3012 and the end of timing pulse #4 of 3014represents the operational ToF range. Within this range all of thereflected light is collected. In contrast, most of the ambient light isoutside the time range and is not collected. A further advantage of thein-pixel differential mode for ToF is that ambient light contributioncan be further reduced or canceled. After the time range for ToFoperation, the photo charges can be drained. In some embodiments, aglobal shutter gate is disposed between the photodiode and a powersupply, and turning on the global shutter gate can drain most of thecollected photo charges. For example, 90% of the photo charges can bedrained from the photodiode. This particular percentage is merelyrepresentative and other percentages are included within the scope ofthe present invention. Although FIG. 30A is primarily described abovewith reference to four timing pulses (e.g., timing pulse #1 3001, timingpulse #2 3002, timing pulse #3 3003, and timing pulse #4 3004), it is tobe understood that, in some implementations, one or more of the imagesensing devices described herein may employ fewer or greater than fourtiming pulses per ToF measurement cycle.

FIG. 30B is a timing diagram illustrating time-of-flight (ToF)operations for the pixel cell of FIG. 11 according to some embodimentsof the invention. The ToF operations for the pixel cell of FIG. 11 isdescribed above in connection to FIGS. 5-11, where the ToF swing isbetween the storage diode (SD) to the Global Shutter (GS). This timingdiagram can be associated with a drain-only pulsed ToF mode ofoperation. In FIG. 30B, curve 3020 represents a light pulse emitted froma light source toward a target for ToF determination. Curve 3021represents an ambient light signal. Timing curve 3022 represents anexposure time window during which the photodiodes are enabled to senselight input for pixel modulation. In this embodiment, a high intensitylight source is used as the emitter in comparison with the ambient lightsignal 3021. As a result, in some embodiments, a small amount of ambientlight is collected in comparison with reflected light. Therefore, only afew emitted light pulses are utilized in this embodiment, one of whichis shown in FIG. 30B. Similar to the description above in connectionwith FIG. 30A, most of the reflected light is collected, and most of theambient light 3021 is rejected. However, since the output signals arenon-differential, no further ambient light cancellation can be realized.Some energy of the laser light emitted from the light source is drained,for example, in some embodiments, light that is received at the pixelcell at times outside the exposure time window.

FIG. 30C is a timing diagram illustrating a conventional phasemodulation time-of-flight (ToF) operation. In FIG. 30C, 3030 representslaser light emitted from the light source, and 3031 represents theambient light. Additionally, 3032 and 3035 represent timing pulses forphase modulation time-of-flight (ToF) determination. For each lightpulse emitted toward a target, the light reflected from the target iscollected in two consecutive timing windows in 3032 and 3035,respectively. The ToF information is derived from the two sensedreflected light signals. In this ToF method, no ambient light isrejected, and all laser energy is collected. Further, differentialoutput can cancel ambient light.

FIG. 31 is a plot of electrical potentials illustrating the operation ofa photodiode in the pixel cell of FIG. 28 according to some embodimentsof the invention. With reference to the pixel cell 2800 of FIG. 28, FIG.31 shows the electrical potentials across photo diode PPD4, storagediodes SD1 and SD4 on either side of PPD4, floating diffusions FD, andpower supply VDD. Transfer signal TX1_2 controls charge transfer fromPPD4 to SD1, and transfer signal TX1_3 controls charge transfer fromPPD4 to SD4. Transfer signals TX2 controls charge transfer from thestorage diodes SD1 and SD3 to the floating diffusion regions FD. A resetsignal, RST, applies the power supply to reset the circuit. As describedabove in connection with the timing plots in FIG. 30A, transfer signalsTX1_1 and TX1_3 are toggled in a complementary fashion, i.e., when oneis on, the other is off, and vice versa. In other words, the transfergate TX1 is split into multiple transfer gates, e.g., TX1_1, TX1_2,TX1_3, and TX1_4, to modulated and non-modulated parts. In FIG. 31, thesolid line illustrates the transfer of photo charges from photo diodePPD4 to storage diode SD4, when transfer signal TX1_3 is turned on. Thedotted line illustrates, when transfer signal TX1_2 is turned on, thetransfer of charges from photo diode PPD4 to storage diode SD1. Thedotted line also shows the transfer of charges from storage diode SD1 tofloating diffusion FD under the control of transfer signal TX2, and thedraining of charges under the control of a reset signal RST. As shown,TX1_2 and TX1_3 toggle in a complementary manner. In other words, whenone of TX1_2 and TX1_3 is higher, the other is low, and vice versa. Thisarrangement can prevent spill-back of charge.

As described above, each of the photodiodes in the pixel cell 2800 ofFIG. 28 is coupled to two storage diodes on either side of each of thephotodiodes. During a first time window, the charges in the photo diodePPD4 are transferred to storage diode SD4, as illustrated by the solidpotential curve in FIG. 31. During a second time window, the charges inthe photo diode PPD4 are transferred to storage diode SD1, asillustrated by the dotted potential curve in FIG. 31A. At this time, thecharges are kept in SD1 and SD4, respectively, for transferring to thefloating diffusion regions in the readout circuits. Charge controlsignal TX3 controls a charge control gate disposed over the storagediode, so as to prevent charge from leaking back to the PPD and to allowreduction of the gate area and therefore the capacitance of TX1_N. Thereduced capacitance and leakage enable faster ToF modulation with highercontrast. Therefore, the pixel cell of FIG. 28 functions as a chargestorage unit to allow for charge collection during different timingwindows, resulting in improvement in the ToF modulation performance. Thepixel cell is also configured to prevent spill-back of charge, reducedgated capacitance (which can affect gating speed and contrast), andimproved modulation contrast. Further, the pixel cell can also enabledifferential and non-differential image capture as described below inconnection with FIGS. 34 and 35. Moreover, the pixel cell describedabove is also compatible with the pinned-photodiode global shutter pixeldesign.

FIGS. 32A and 32B are plots of electrical potentials illustrating theoperation of a photodiode in the pixel cell of FIG. 11 according to someembodiments of the invention. In the pixel cell of FIG. 11, each photodiode (PPD) is coupled to only one storage diode (SD). In FIG. 31,turning on the transfer gate TX1 transfers photo charges from the photodiode (PPD) to the storage diode (SD). The photo charges are thentransferred to a readout circuit. In FIG. 32A, after an exposure period,turning on the global shutter (GS) drains the charges in the photodiode. Therefore, the photo diode is no longer available to providefurther photo charges.

In some embodiments of the invention, pixel cells depicted in FIGS.28-31 can be used in an image sensor device for ToF distancemeasurement. The image sensing device can include a plurality of pixelcells arranged in a matrix in a pixel array and a control circuit forcontrolling an exposure phase and a sampling phase of the image sensordevice. FIGS. 2A and 2B illustrate examples of a plurality of pixelcells arranged in a matrix in a pixel array.

In the pixel cells depicted in FIGS. 28-31, each pixel cell includesfour photodiodes, four storage diodes, and four floating diffusionregions. Each storage diode is disposed between a first adjacentphotodiode and a second adjacent photodiode, and each storage diode isconfigured to receive photo charges from either or both of the firstadjacent photodiode and/or the second adjacent photodiode. Further, eachphotodiode is disposed between a first adjacent storage diode and asecond adjacent storage diode, and each photodiode is configured totransfer photo charges to either or both of the first adjacent storagediode and/or the second adjacent storage diode. Further, each floatingdiffusion region is disposed adjacent to a corresponding storage diode.

In alternative embodiments, the pixel cell is not limited to fourphotodiodes. In general, a pixel cell can include a plurality ofphotodiodes and a corresponding plurality of storage diodes. The methodsdescribed herein can be adopted to pixels having more than four pixelphotodiodes. Similarly, in some embodiments, the methods describedherein can be adopted to pixels that have fewer than four photodiodes(e.g., two photodiodes or three photodiodes). Furthermore, in someembodiments, the pixel cell is not limited to four storage diodes. Themethods described herein can be adopted to pixels having more or fewerthan four storage diodes. In some implementations, a pixel cell caninclude unequal quantities of photodiodes and storage diodes. Forexample, in such implementations, a pixel cell may include an oddquantity of photodiodes (e.g., five photodiodes) and an even quantity ofstorage diodes (e.g., four storage diodes) arranged in acheckerboard-like configuration. In this example, each storage diode maybe disposed between more than two photodiodes (e.g., three photodiodesor four photodiodes) and also configured to receive photo charges frommore than two photodiodes. One of ordinary skill in the art wouldrecognize many variations, modifications, and alternatives.

As shown in FIGS. 28 and 29B, the image sensor device also includes afirst summing device for receiving photo charges from a second floatingdiffusion region and a fourth floating diffusion region, and a secondsumming device for receiving photo charges from a first floatingdiffusion region and a third floating diffusion region. As shown in FIG.29B, the image sensor device can also include a differential amplifiercoupled to the first and second summing devices. The image sensor devicecan also have a control circuit for controlling charge transfer in theimage sensing device. The control circuit can include circuitry such aspulsed illumination unit 110, sensor unit 130, and ToF timing generator150 as illustrated in FIGS. 1, 3, 4A and 4B.

In some embodiments, the image sensor device can be used fordifferential ToF measurement. As described in connection with FIGS.28-31, the control circuit in the image sensor device is configured forimplementing a method in differential ToF mode.

FIG. 33 is a flowchart summarizing a method 3300 for a differential ToFmode operation. The method starts with, at 3310, exposing a pixel cellto a light during an exposure time window. At 3320, during a first timeperiod, the collected photo charges are transferred from a first pair ofphotodiodes to a first storage diode disposed between the first pair ofphotodiodes, and, at 3330, the collected photo charges from a secondpair of photodiodes are transferred to a second storage diode disposedbetween the second pair of photodiodes.

During a second time period, at 3340, the collected photo charges aretransferred from a third pair of photodiodes to a third storage diodedisposed between the third pair of photodiodes, and, at 3350, thecollected photo charges are transferred from a fourth pair ofphotodiodes to a fourth storage diode disposed between the fourth pairof photodiodes. At 3360, a first sum of the photo charges from thesecond storage diode and the fourth storage diode is formed. At 3370, asecond sum of the photo charges from the first storage diode and thethird storage diode is formed. At 3380, the first sum is transferred toa first input of the differential amplifier, and the second sum istransferred to a second input of the differential amplifier. At 3390,the differential amplifier determines a differential signal based on thefirst sum and the second sum. The differential signal can be used in ToFdetermination.

It should be appreciated that the specific steps illustrated in FIG. 33provide a particular method of performing differential ToF modeoperation according to an embodiment of the present invention. Othersequences of steps may also be performed according to alternativeembodiments. For example, alternative embodiments of the presentinvention may perform the steps outlined above in a different order.Moreover, the individual steps illustrated in FIG. 33 may includemultiple sub-steps that may be performed in various sequences asappropriate to the individual step. Furthermore, additional steps may beadded or removed depending on the particular applications. One ofordinary skill in the art would recognize many variations,modifications, and alternatives.

In some embodiments, a method for operating a pixel cell in adifferential ToF mode can include exposing a pixel cell to a lightduring an exposure time window, the pixel cell including fourphotodiodes and four storage diodes, as described above. The methodincludes, during a first time period, transferring collected photocharges from a first pair of photodiodes to a first storage diodedisposed between the first pair of photodiodes, and transferringcollected photo charges from a second pair of photodiodes to a secondstorage diode disposed between the second pair of photodiodes. Themethod includes, during a second time period, transferring collectedphoto charges from a third pair of photodiodes to a third storage diodedisposed between the third pair of photodiodes, and transferringcollected photo charges from a fourth pair of photodiodes to a fourthstorage diode disposed between the fourth pair of photodiodes. Themethod also includes producing a differential signal by providing a sumof the photo charges from the first storage diode and the second storagediode to a first input of a differential amplifier, and providing a sumof the photo charges from the third storage diode and the fourth storagediode to a second input of the differential amplifier.

In some embodiments, the method can also include transferring photocharges from the first storage diode and the second storage diode to afirst floating diffusion region, and transferring photo charges from thefirst storage diode and the second storage diode to a second floatingdiffusion region. The method can also include transferring photo chargesfrom the first floating diffusion region to a first sample-and-holdcapacitor, transferring photo charges from the second floating diffusionregion to a second sample-and-hold capacitor, and transferring signalsfrom the first and second sample-and-hold capacitors to the differentialamplifier. In some embodiments of the method, the first pair ofphotodiodes and the second of pair photodiodes have no photodiode incommon, and the third pair of photodiodes and the fourth pair ofphotodiodes have no photodiode in common. An example of the method fordifferential ToF mode is described in connection to FIGS. 28-31.

In some embodiments, a method for operating a pixel cell in a binningmode can include transferring collected photo charges in a first pair ofadjacent photodiodes to a first storage diode, and transferringcollected photo charges in a second pair of adjacent photodiodes to asecond storage diode. The method can also include sensing photo chargesin the first storage diode and the second storage diode to provide twosensed signals for binning. An example of the method for the binningmode is described in connection to FIG. 34.

In some embodiments, a method for operating a pixel cell in a fullresolution mode can include transferring collected photo charges in eachphotodiode to an adjacent storage diode, and sensing photo charges ineach storage diode to provide sensed signals for four sub-pixels. Anexample of the method for the full resolution mode is described inconnection to FIG. 35.

As shown in FIG. 28, pixel cell 2800 includes four photodiodes PPD1,PPD2, PPD3, and PPD4. Pixel cell 2800 also includes four storage diodes,which are disposed under the four storage diode charge control gates. Itcan be seen that each storage diode is disposed between a first adjacentphotodiode and a second adjacent photodiode, and each storage diode isconfigured to receive photo charges from either or both of the firstadjacent photodiode and/or the second adjacent photodiode. Further, eachphotodiode is disposed between a first adjacent storage diode and asecond adjacent storage diode, and each photodiode is configured totransfer photo charges to either or both of the first adjacent storagediode and/or the second adjacent storage diode under the control of thetransfer gates. The charge transfers between photo diodes and storagediodes are under the control of the transfer gates TX1_1, TX1_2, TX1_3,and TX1_4, as described above in connection with FIG. 28.

In the embodiments of FIGS. 28-29B, the four photodiodes are arranged ina 2-by-2 array, with each storage diode disposed between two adjacentphotodiodes. There is a transfer gate between each pair of adjacentphotodiode and storage diode. Pixel cell 2800 also includes fourfloating diffusion regions, similar to pixel cell 28 in FIG. 28. Eachfloating diffusion regions is disposed adjacent to corresponding storagediode. One floating diffusion region FD1 is shown. There is a transfergate between each pair of adjacent storage diode and floating diffusionregion. There is also a charge control gate TX3 overlying each storagediode, and configured to maintaining a constant voltage.

In some embodiments of the invention, pixel cells depicted in FIGS.28-29B can be used in an image sensor device for ToF distancemeasurement. The image sensing device can include a plurality of pixelcells arranged in a matrix in a pixel array and a control circuit forcontrolling an exposure phase and a sampling phase of the image sensordevice. FIGS. 2A and 2B illustrate examples of a plurality of pixelcells arranged in a matrix in a pixel array. The control circuit caninclude circuitry such as pulsed illumination unit 110, sensor unit 130,and ToF timing generator 150 as illustrated in FIGS. 1, 3, and FIGS. 4Aand 4B.

FIG. 34 is a simplified top view schematic diagram illustrating a pixelcell for binning mode operation according to some embodiments of theinvention. FIG. 34 illustrates a pixel cell 3400 that includes fourphotodiodes and four storage diodes, similar to the pixel cells of FIGS.28-29B. Each storage diode is disposed between two adjacent photodiodes,and each storage diode is configured to receive photo charges fromeither or both of the two adjacent photodiodes. Each photodiode isdisposed between two adjacent storage diodes, and each photodiode isconfigured to transfer photo charges to either or both of the twoadjacent storage diodes. For a binning mode of operation, the collectedphoto charges in a first pair of adjacent photodiodes are transferred toa first storage diode, and the collected photo charges in a second pairof adjacent photodiode are transferred to a second storage diode. Forexample, in FIG. 34, the collected photo charges in photodiodes PPD2 andPPD3 are transferred to storage diode SD3, and the collected photocharges in photodiodes PPD1 and PPD4 are transferred to storage diodeSD1. Then, photo charges in the first storage diode (SD3) and the secondstorage diode (SD1) are sensed to provide two sensed signals forbinning. In this example, transfer signals TX1_1 and TX1_2 are toggledtogether to control charge transfer. The operation described above issimilar to the operation taking place during timing pulse #1 in FIG. 28.Alternatively, different pairing of photo diodes can also be used forthe binning operation. For example, similar to the operation takingplace during timing pulse #2 in FIG. 28, the collected photo charges inphotodiodes PPD1 and PPD2 are transferred to storage diode SD2, and thecollected photo charges in photodiodes PPD3 and PPD4 are transferred tostorage diode SD4. Then, photo charges in the first storage diode (SD2)and the second storage diode (SD4) are sensed to provide two sensedsignals for binning. In this example, transfer signals TX1_3 and TX1_4are toggled together to control charge transfer. As described below withreference to FIG. 37C, the binning mode can be used in a binneddifferential mode ToF determination.

FIG. 35 is a simplified top view schematic diagram illustrating a pixelcell for a full resolution mode operation according to some embodimentsof the invention. FIG. 35 illustrates a pixel cell 3500 that includesfour photodiodes and four storage diodes, similar to the pixel cells ofFIGS. 28-29B. Each storage diode is disposed between two adjacentphotodiodes, and each storage diode is configured to receive photocharges from either or both of the two adjacent photodiodes. Eachphotodiode is disposed between two adjacent storage diodes, and eachphotodiode is configured to transfer photo charges to either or both ofthe two adjacent storage diodes. For a full resolution mode operation,the collected photo charges in each photodiode are transferred to arespective adjacent storage diode. The photo charges in each storagediode is sensed to provide sensed signals for four sub-pixels. Forexample, in FIG. 35, the collected photo charges in photodiodes PPD1 aretransferred to storage diode SD2, the collected photo charges inphotodiodes PPD2 are transferred to storage diode SD3, the collectedphoto charges in photodiodes PPD3 are transferred to storage diode SD4,and the collected photo charges in photodiodes PPD4 are transferred tostorage diode SD1. Then, photo charges in each of the four storagediodes, SD1, SD2, SD3, and SD4 are sensed to provide four sensed signalsfor each of the four sub-pixels. In this example, transfer signals TX1_2and TX1_4 can be toggled together to control charge transfer. FIG. 35also shows four read out circuits, RO_1, RO_2, RO_3, and RO_4, eachhaving a floating diffusion region for reading out the optical signals.

Alternatively, different associations of photodiodes and storage diodescan also be used for the full resolution mode operation. For example, inFIG. 35, the collected photo charges in photodiodes PPD1 can betransferred to storage diode SD1, the collected photo charges inphotodiodes PPD2 can be transferred to storage diode SD2, the collectedphoto charges in photodiodes PPD3 can be transferred to storage diodeSD3, and the collected photo charges in photodiodes PPD4 can betransferred to storage diode SD4. Then, photo charges in each of thefour storage diodes, SD1, SD2, SD3, and SD4 are sensed to provide foursensed signals for each of the four sub-pixels.

FIG. 36 is line drawing plot illustrating the layout of a portion of thepixel cell of FIG. 28 according to some embodiments of the invention.Referring to the timing diagram in FIG. 30A, in differential ToF mode,the two swinging phases, as shown by labels Storage 1 and Storage 2 asshown in FIG. 31A, are interleaved between two rows of pixels to reducefringe capacitance. The two metal lines 3610 and 3612 associated withlabel Storage 1, e.g., metal lines associated with transfer signalsTX1_3 and TX1_4 in FIG. 28, are toggled together. Therefore, thecapacitance between the two lines is low. Similarly, the two metal lines3620 and 3622 associated with label Storage 2, e.g., metal linesassociated with transfer signals TX1_1 and TX1_2 in FIG. 28, are toggledtogether. Therefore, the capacitance between the two lines is low. Onthe other hand, the two metal lines associated with label Storage 1 aredisposed away from the two metal lines associated with label Storage 2.This arrangement can reduce the capacitance even with thick metal linesto reduce electrical resistance. This arrangement is desirable for ToFoperations, when fast responses are required, e.g., in the range of subnano-seconds.

FIGS. 37A, 37B, and 37C are simplified timing diagrams illustratingdifferent modes of operation that can be implemented using the pixelcell of FIGS. 28-29B according to some embodiments of the invention.FIG. 37A illustrates a timing diagram for a standard image sensordifferential mode, in which two full readouts are required. In thismode, the first pulse is operated together with active illumination, andthe second is operated when the active illumination is at “OFF” state.In this differential mode, the sensor works in a binned mode, and a fullreadout is performed between captures. The differential signal canobtained by a subtraction operation digitally between captured frames.

FIG. 37B illustrates a timing diagram for in-pixel differential ToFmode, as illustrated above in connection with FIGS. 28-36, in which onlyone differential readout is performed. In this mode, there is no readoutsequence between the active illuminated sub-frame and the OFF-statesub-frame. In this in-pixel differential mode, the sensor operates indifferential mode, and differential signals are generated by subtractionperformed using the differential amplifier.

FIG. 37C illustrates a timing diagram for an advanced “dynamic mode”operation according to some embodiments of the invention. In the dynamicmode, the array of pixel cells are scanned with differentialcomparators, addresses of pixels with sensed signals above a presetthreshold are transmitted via a communication link, such as SPI (SerialPeripheral Interface). In this mode, selective, event-based output canbe provided, with low data bandwidth. In some embodiments, activeillumination is not required, and changes in the signal with the timeare tracked, by alternately integrating to the two storages. In someembodiments, each pixel has two storages, and an image can be comparedwith a previous image. The differences can be tested by a rollingshutter and a simplified ADC (analog-to-digital converter), that caninclude only a single bit. It can be implemented with a comparator.

While the preferred embodiments of the invention have been illustratedand described, it will be clear that the invention is not limited tothese embodiments only. Numerous modifications, changes, variations,substitutions and equivalents will be apparent to those skilled in theart without departing from the spirit and scope of the invention asdescribed in the claims.

What is claimed is:
 1. A pixel cell, comprising: four photodiodes; andfour storage diodes; wherein each storage diode is disposed between afirst adjacent photodiode and a second adjacent photodiode, and eachstorage diode is configured to receive photo charges from either or bothof the first adjacent photodiode and the second adjacent photodiode;wherein each photodiode is disposed between a first adjacent storagediode and a second adjacent storage diode, and each photodiode isconfigured to transfer photo charges to either or both of the firstadjacent storage diode and the second adjacent storage diode; andwherein the four photodiodes are arranged in a 2 by 2 array, with eachstorage diode disposed between two adjacent photodiodes.
 2. The pixelcell of claim 1, further comprising a transfer gate between each pair ofadjacent photodiode and storage diode.
 3. The pixel cell of claim 1,further comprising a charge control gate overlying each storage diode.4. The pixel cell of claim 1, further comprising four floating diffusionregions, each floating diffusion regions disposed adjacent to acorresponding storage diode.
 5. The pixel cell of claim 4, furthercomprising a transfer gate between each pair of adjacent storage diodeand floating diffusion region.
 6. The pixel cell of claim 1, furthercomprising a global shutter control gate associated with each photodiodefor draining the charges in the photodiode.
 7. An image sensor device,comprising a plurality of pixel cells of claim 1 arranged in a pixelarray.
 8. A method for operating a pixel cell, the method comprising:exposing a pixel cell to a light during an exposure time window, thepixel cell including four photodiodes and four storage diodes; whereineach storage diode is disposed between two adjacent photodiodes in thepixel cell, and each storage diode is configured to receive photocharges from either or both of the two adjacent photodiodes; whereineach photodiode is disposed between two adjacent storage diodes in thepixel cell, and each photodiode is configured to transfer photo chargesto either or both of the two adjacent storage diodes; and wherein eachpair of adjacent photodiodes has a storage diode disposed therebetween.9. The method of claim 8, further comprising: during a first timeperiod, transferring collected photo charges from a first pair ofphotodiodes to a first storage diode disposed between the first pair ofphotodiodes; and transferring collected photo charges from a second pairof photodiodes to a second storage diode disposed between the secondpair of photodiodes; during a second time period, transferring collectedphoto charges from a third pair of photodiodes to a third storage diodedisposed between the third pair of photodiodes; and transferringcollected photo charges from a fourth pair of photodiodes to a fourthstorage diode disposed between the fourth pair of photodiodes; andproducing a differential signal by: providing a sum of the photo chargesfrom the first storage diode and the second storage diode to a firstinput of a differential amplifier; and providing a sum of the photocharges from the third storage diode and the fourth storage diode to asecond input of the differential amplifier.
 10. The method of claim 9,further comprising: transferring photo charges from the first storagediode and the second storage diode to a first floating diffusion region;transferring photo charges from the first storage diode and the secondstorage diode to a second floating diffusion region; transferring photocharges from the first floating diffusion region to a firstsample-and-hold capacitor; transferring photo charges from the secondfloating diffusion region to a second sample-and-hold capacitor; andtransferring signals from the first and second sample-and-holdcapacitors to the differential amplifier.
 11. The method of claim 9,wherein; the first pair of photodiodes and the second pair ofphotodiodes have no photodiode in common; and the third pair ofphotodiodes and the fourth pair of photodiodes have no photodiode incommon.
 12. The method of claim 8, further comprising: transferringcollected photo charges in a first pair of adjacent photodiodes to afirst storage diode; transferring collected photo charges in a secondpair of adjacent photodiodes to a second storage diode; and sensingphoto charges in the first storage diode and the second storage diode toprovide two sensed signals for binning.
 13. The method of claim 8,further comprising: transferring collected photo charges in eachphotodiode to a respective adjacent storage diode; and sensing photocharges in each storage diode to provide sensed signals for foursub-pixels.
 14. The method of claim 8, wherein the four photodiodes arearranged in a 2 by 2 array, with each storage diode disposed between twoadjacent photodiodes.
 15. A pixel cell for differential light sensing,comprising: a plurality of photodiodes; a corresponding plurality ofstorage diodes; and a corresponding plurality of floating diffusionregions, each floating diffusion region disposed adjacent to acorresponding storage diode; wherein each storage diode is disposedbetween a first adjacent photodiode and a second adjacent photodiode,and each storage diode is configured to receive photo charges fromeither or both of the first adjacent photodiode and the second adjacentphotodiode; and wherein each photodiode is disposed between a firstadjacent storage diode and a second adjacent storage diode, and eachphotodiode is configured to transfer photo charges to either or both ofthe first adjacent storage diode and the second adjacent storage diode.16. An image sensing device, comprising: a plurality of pixel cellsarranged in a pixel array, wherein each pixel cell includes fourphotodiodes, four storage diodes, and four floating diffusion regions,wherein: each storage diode is disposed between a first adjacentphotodiode and a second adjacent photodiode, and each storage diode isconfigured to receive photo charges from either or both of the firstadjacent photodiode and the second adjacent photodiode; each photodiodeis disposed between a first adjacent storage diode and a second adjacentstorage diode, and each photodiode is configured to transfer photocharges to either or both of the first adjacent storage diode and thesecond adjacent storage diode; and each floating diffusion region isdisposed adjacent to a corresponding storage diode; a first summingdevice for receiving photo charges from a second floating diffusionregion and a fourth floating diffusion region; a second summing devicefor receiving photo charges from a first floating diffusion region and athird floating diffusion region; a differential amplifier coupled to thefirst and second summing devices; and a control circuit for controllingcharge transfer in the image sensing device.
 17. The image sensingdevice of claim 16, wherein the control circuit is configured for:exposing a pixel cell to a light during an exposure time window; duringa first time period, transferring collected photo charges from a firstpair of photodiodes to a first storage diode disposed between the firstpair of photodiodes; and transferring collected photo charges from asecond pair of photodiodes to a second storage diode disposed betweenthe second pair of photodiodes; during a second time period,transferring collected photo charges from a third pair of photodiodes toa third storage diode disposed between the third pair of photodiodes;and transferring collected photo charges from a fourth pair ofphotodiodes to a fourth storage diode disposed between the fourth pairof photodiodes; and producing a differential signal by: providing a sumof the photo charges from the second storage diode and the fourthstorage diode to a first input of the differential amplifier; andproviding a sum of the photo charges from the first storage diode andthe third storage diode to a second input of the differential amplifier.18. The image sensing device of claim 16, wherein the control circuit isconfigured for: exposing a pixel cell to a light during an exposure timewindow; transferring collected photo charges in a first pair of adjacentphotodiodes to a first storage diode; transferring collected photocharges in a second pair of adjacent photodiode to a second storagediode; and sensing photo charges in the first storage diode and thesecond storage diode to provide two sensed signals for binning.
 19. Theimage sensing device of claim 16, wherein the control circuit isconfigured for: exposing a pixel cell to a light during an exposure timewindow; transferring collected photo charges in each photodiode to acorresponding adjacent storage diode; and sensing photo charges in eachstorage diode to provide sensed signals for four sub-pixels.